A tiny piece of cheese won't hurt most cats. The problem is that most adult cats are lactose intolerant — they lack sufficient lactase enzyme to properly digest dairy, which means regular cheese feeding leads to gas, diarrhea, and vomiting. A nibble that falls on the floor? Fine. A piece of cheese as a daily treat? Not a great idea.
Cheese is also high in fat and sodium, which are additional reasons not to make it a regular thing for a small animal.
Why cats and dairy don't mix well
It's a common cultural image — cats lapping at a bowl of milk — but the reality is that adult cats handle dairy poorly. Like most mammals, cats are born able to digest their mother's milk, but as they mature they produce less and less lactase (the enzyme that breaks down lactose). By adulthood, most cats have very little of it left.
Cheese has lower lactose than liquid milk because some of the lactose is converted during fermentation, which is why some lactose-intolerant cats can handle a small amount of cheese without obvious symptoms. But lower isn't zero, and individual cats vary in how sensitive they are.
If your cat begs for cheese
A very small piece occasionally — the size of a pea — is unlikely to cause any harm in most cats. The issue is if they show interest in dairy, pet owners tend to give more than that, and it accumulates. If your cat gets stomach upset after cheese even in small amounts, they're probably quite lactose intolerant and you should drop it entirely.
For a protein-based treat your cat will love without the dairy issues, plain cooked chicken is far better suited to a cat's biology. Or plain cooked egg — both give you the high-protein reward without involving lactose at all.
The whole cats-love-milk thing is really just a cultural myth that doesn't reflect what's actually good for them.