Nigger: 1) Usually offensive to people of the black race.
2) A member of a socially disadvantaged class of persons.
3) All of the people that feel left out of the political process.
In this poem the Baltimorean is calling the boy a nigger because he is different. He is a different color so in that time that would make him bad. The Baltimorean is calling him that with out even knowing him.
Infomation found at: http://www.m-w.com/cgi-bin/dictionary