The Solana network encountered a major outage on February 6th at 10:22 am UTC, resulting in block production being halted for over five hours until validators were able to restart the network. This is not the first time Solana has experienced downtime, as there have been several significant outages and partial or major outage days since January 2022.
These incidents varied in severity, ranging from partial to total network outages, and were caused by software vulnerabilities, denial-of-service attacks, and resource exhaustion attacks. The most recent outage was attributed to a bug that caused an infinite loop error, which prevented validators from verifying transactions on the network.
Anza, a Solana-focused software development firm, released a postmortem report on February 9th, providing details on the cause of the outage and how the network was restarted. The report revealed that the bug occurred in Solana’s Just-in-Time (JIT) compilation cache, which encountered a loop with older programs due to a legacy loader issue. This caused the system to become stuck in an infinite loop and unable to process transactions.
Solana engineers quickly identified and fixed the problem, updated the system’s processes, and implemented measures to prevent future recompilation cycles. They reduced the deployment of troublesome legacy loaders to ensure the accurate and efficient access and execution of program instructions.
Critics have pointed out that Solana’s lack of client diversity may contribute to its outages, leading to potential central points of failure. However, Austin Federa, head of strategy at the Solana Foundation, stated that Solana’s architecture is accurate and that the network has never experienced an outage due to a fault in consensus. He emphasized that the issues have always been implementation details.
Although Solana has achieved significant milestones in the decentralized finance (DeFi) ecosystem and has a large market valuation, it is still referred to as being in mainnet beta. The blockchain has a vibrant ecosystem with numerous decentralized applications and a thriving nonfungible token (NFT) ecosystem.
Despite its beta status and past outages, Solana’s developers are actively working to ensure the network’s stability and attract new projects with its high speed and low cost advantages.