A childhood belief about misheard lyrics

When my sister and I were younger we loved the song Mellow Yellow by Donavan.I think initially we liked it because we thought he was singing about the soda. After we heard it a few times though we thought the guy was really cool for a couple of reasons
1)There was this sidekick guy that agreed with everything he said by saying Quite right slick.
and
2)This girl named Saffron seemed to really dig him which we decided was because at some point in the song he self proclaimed that he had a big fat banana.
Funny though as an adult I can't ever hear the part of the song where he says.
I have a big fat banana.
But I swear it was there when I was younger followed by his sidekick saying:
Quite right slick.