Do longer focal lengths affect the amount of light that reaches the censor, versus shorter zoom lenses ?

My current lens is the 24-105 mm L (canon) and I understand that the it's maximum apeture is 4.0, and that with the 24-70 mm L lens, that it's maximum apeture is 2.8. But lets just say I was shoot a picture with both lenses at an apeture of 4.0 - would the photo shot with the lens with the shorter focal length be brighter than the one shot with the lens with the longer focal length ? Does this long focal length affect the amount of light reaching the censor and thefore result in darker photos ? Should I purchase a shorter zoom lens or prime lens ?