Is "poor" an insulting term?

This is a small personal anecdote from many years back: I used to be a landlord letting out a few houses that I owned at the time. On one occasion I had a house vacant and put a "To Let" advertisement in the newspaper. The advertisement stated the amount of the rent, the suburb, a brief description of the house and a statement that a bond equivalent to four week's rent was required in addition to the rent before a tenant could move in. All of which was customary in that time and place.

One caller said he liked the sound of the house but that he wouldn't have all the money needed until payday so could I hold it for him until then?

There was something irritating in his approach, however, a sort of arrogance. So I said, No, I didn't want poor people in my house.

He was outraged and claimed that he was not poor -- even though he had no money. He went on to abuse me and even called back later to abuse me again. I guess my frankness was a bit brutal but how much better it would be if people could face and accept reality. I may in fact have been instrumental in getting him to start saving his money.