The pilot reported the wind was from the north-northwest at 6 knots when they departed and the balloon traveled to the southeast as expected. When they descended to land the wind velocity increased to 20 plus knots and the wind changed 90 degrees in direction moving them to the west. The pilot attempted to land twice and aborted both landings due to the wind and terrain conditions. The pilot briefed the passenger, who was very familiar with balloon operations, that they were going to make a high wind landing. The passenger stated he crouched down in the basket for the landing and as the triangular basket touched down and turned to its flat side, he felt his ankle roll. The basket lay down and once the balloon stopped they crawled out of the basket. The passenger stated he knew he injured his ankle and a couple days later he was diagnosed with a fractured tibia.