I think what Ivan was getting at was his distaste for the "thought policing" inherent in hate crimes. Ivan was saying the intent shouldn't matter, the action or crime should.
Intent, though, is an element of every crime. The reason someone engages is a potential criminal act always matters.
For example, let's say you saw a bike that looked exactly like yours and you took it thinking it was your bike. Turns out it was someone else's. But you never formed the intent to steal so your not guilty of anything. Intent to steal would have to be proven to convict you of theft.
In a hate crime, if the intent to harm someone because they are gay or black or whatever can be shown, a harsher punishment is appropriate, because that is worse than just beating someone up because they called you an asshole.