A poll found that one in five get so anxious about having to do maths, it makes them feel physically sick.