Yes, I get very irritated, especially by lateness. It is very discourteous, I think, for someone with whom one has made a specific arrangement to arrive late. It implies that they consider their own time and convenience to be of greater importance than mine. Of course, there can be extenuating circumstances when some emergency or genuinely unforeseeable event intervenes. But otherwise, I make it a point of principle to be punctual myself, and I think others should too. This usually requires some sacrifices on my part to make sure that I am not late, and I don't see why some other people won't make the same sacrifices too.
I sometimes like to invite a group of people to dinner at my house, and I hate it when there is one person who shows up substantially late, with no good excuse. Sometimes timings of the cooking of dishes can be critical, and it can throw out the entire schedule, possibly ending with a dish being spoiled, if one person arrives late. When I am invited to an event for 7:00pm, or whatever, I show up at 7:00pm on the dot, and I wish others would do the same.