Reading newspaper at late lunch?

I work at a restaurant where am permitted to eat for lunch. Had late lunch today and senior colleague saw me reading the newspaper whilst waiting for my food and said "nice read?"..I just said yes. Should I refrain from reading paper there in future...or am I worrying too much?