Do i give in or hang back?

i just recently moved to a new school and were immediatly befriended by the 'populars' and they wanted me to ditch homeroom and sneak off campus to get some lunch, i dont want to be a party pooper but im nervous ill get caught, would should i do??!