Tower Controller reported a departing aircraft turned off the SID and flew below the minimum vectoring altitude.

Narrative

Aircraft X departed ZZZ on the ZZZ Standard Instrument Departure (SID). Aircraft X turned off the SID placing him in a high Minimum Vector Area (MVA). I advised the pilot he was not doing what the SID said; asked if he had the terrain in sight which the pilot advised he had terrain in sight and issued a Low Altitude Alert. I did not advise the pilot how high the MVA was. I had the aircraft in sight so I issued the aircraft a vector to get him back on course. I did not submit a MOR (Mandatory Occurrence Report); because I thought it was a quick spot correction; however Management advised that it should have been a significant MOR event. Additionally management advised me on the rules and how I could have accomplished the same thing a lot cleaner adhering to the specifics of the rules.I have went over the appropriate procedures and course of action I should have taken in this scenario with Management. A recommendation is possibly having scenarios like this once or twice a year to brush up on executing these procedures and adhering to the rules appropriately.