Removing e92 gear knob

I bought a msport gear knob and handbrake handle. Have changed the handbrake but for the life of me i cant get the se gear off.



Have searched and it seems that you are supposed to put gear in 2nd and pull the knob upwards. Have been trying for almost an hour but cant get it to budge even a tiny bit.



Anyone have any advice? Is it really just slide off same as the handbrake?



Thanks in advance