The pilot reported that during the landing roll, the tailwheel-equipped airplane began to “fishtail” and drift toward the side of the runway. The pilot attempted to correct the oscillation by applying the brakes, but was “slightly late” in applying the correction and the airplane departed the side of the runway. After becoming partially buried in the soft ground, the right main landing gear collapsed, resulting in substantial damage to the fuselage at the right main landing gear attach point as well as the engine firewall. The pilot reported there were no mechanical malfunctions or anomalies that would have precluded normal operation of the airplane.