PA-28 pilot reported an engine malfunction during climb resulting in loss of RPM and roughness on an IFR flight plan in visual conditions on a training flight. The flight diverted to an alternate airport and landed.

Narrative

I was flying a Piper P28A Archer. We were flying from ZZZ to ZZZ1. During the climb through 5;000 to 6;000 feet--actually; we were climbing to 8;000 feet--and on the climb through 6;000 feet; I started to notice a gradual reduction in RPM. As we continued; I noticed that the VSI was decreasing at the same time the RPM was rolling back. We stopped climbing; we requested to descend to 6;000; and we turned on the carb heat thinking maybe it was carburetor icing. We left it on for over five minutes to ensure that any carburetor icing that was accumulated would have been gone. The mixture was also properly leaned for the altitude we were flying at.As we leveled off at 6;000 feet and I tried to regain RPM; I asked the Approach Controller for a block altitude to try and re-diagnose the issue. As we attempted the second climb; the same thing happened but worse. The lowest the RPM went at altitude was 2;000; experiencing a total 500 RPM drop. We then told the Approach Controller that we needed to divert to ZZZ2 and wouldn't be able to continue to ZZZ1.On the approach to land through the descent; we coordinated the throttle and the mixture accordingly as the engine began running increasingly rough. We approached the runway high to ensure we were in glide distance. As I pulled the power back to 1;400 RPM for landing; we noticed more rough running. As we increased the power to level off; we experienced a temporary significant RPM drop down to 1;200 and then back up to the 1;800 we needed. We tried to troubleshoot in the air and ran the engine roughness checklist and the engine power loss checklist in flight; but we weren't able to come to a solution.We diverted. As we were descending; the engine continuously stuttered and ran rough. On the approach to land; we forward slipped and made a right 360 to ensure we had the field made before reducing the power to idle. When the power was reduced to idle; the engine was extremely abnormal; stuttering very hard. Once we landed and taxied off the runway; the engine was running rough and sputtering all the way to the FBO.I told the Approach Controller that we were not declaring an emergency; however; the Tower provided priority handling for us. They asked for souls on board and fuel remaining and the nature of the situation. There were fire trucks and equipment waiting for us on the ground and they followed us to the FBO. I provided my information to the fire department and called the Tower at ZZZ2 and asked them if they would like a report from me and they said no.After we parked the plane at the FBO; our school's Dispatch contacted a local maintenance facility at the airport and the plane was delivered over there. We found out that the cause of the partial engine failure was due to multiple issues with multiple spark plugs; one of which had so much lead and rust accumulated on it that it begged the question of how long it had been since it was changed. Also; it's fair to note that the airplane had just come out of a 100-hour inspection and had only accumulated 34 hours on the tach timer. The maintenance facility at the airport changed that spark plug on the number 4 cylinder. The maintenance facility also discovered on cylinder number 1 that the spark plug was extremely loose and not operating at full capacity. An additional spark plug also had a broken insulator. The ground check at the maintenance facility initially showed a 350/400 RPM drop on the left Mag. In our run up prior to departing ZZZ there were no abnormalities during our engine run up.After being shown the condition of the airplane by the maintenance facility at ZZZ2 I am severely question the quality and honesty of the A&P and Maintenance department at my school. The situation was entirely avoidable had the maintenance department completed the inspection correctly because they would have noticed the deficiencies. The A&P at the ZZZ2 maintenance facility gave the appropriate logbook entries returning the plane to an airworthy condition. I flew it back with my co pilot the following day without incident. Prior to departing the ZZZ2 airspace I completed 2 traffic patterns to ensure the airplane was flying correctly which it was.