This paper updates the influence of environmental and source factors of shocks driven by coronal mass ejections (CMEs) that are likely to influence the solar energetic particle (SEP) events. The intensity variation due to CME interaction reported that is confirmed by expanding the investigation to all the large SEP events of solar cycle 23. The large SEP events are separated into two groups, one associated with CMEs running into other CMEs, and the other with CMEs running into the ambient solar wind. SEP events with CME interaction generally have a higher intensity. New possibilities such as the influence of coronal holes on the SEP intensity are also discussed. For example, the presence of a large coronal hole between a well-connected eruption and the solar disk center may render the shock poorly connected because of the interaction between the CME and the coronal hole. This point is illustrated using the 2004 December 3 SEP event delayed by about 12 hours from the onset of the associated CME. There is no other event at the Sun that can be associated with the SEP onset. This event is consistent with the possibility that the coronal hole interaction influences the connectivity of the CMEs that produce SEPs, and hence the intensity of the SEP event.