We have measured nonlinear scattering from plasmons in individual Au nanorods and have correlated second-harmonic activity of Ag nanoparticles and clusters to morphology. The measurements reveal novel ultrafast nonlinear phenomena related to electron confinement. Surprisingly, the coherent plasmon response is suppressed relative to the hot electron response indicating enhanced plasmon dephasing. In a parallel set of studies we demonstrate nanometer scale localization of the nonlinear optical response of single nanoparticles and aggregates and correlate this with their morphology. Position markers are fabricated on an optical and electron-transparent substrate (Si 3N4 thin film) that allows optical measurements and transmission electron microscopy (TEM) imaging of the identical nanoparticles or aggregates. The second harmonic (SH) activity optical image of individual Ag nanostructures is registered with the TEM image. Centroid localization of the optical signals allows correlation with better than 25 nm precision. This is sufficient to determine the origin of optical "hot spots" within multi-particle aggregates.