Last Month in Internet Intelligence: January 2019
During the second half of 2018, the causes of significant Internet disruptions observed through the Oracle Internet Intelligence Map could be clustered into a few overarching areas: government-directed, cable problems, power outages, distributed denial of service (DDoS) attacks, and general technical issues. Little changed heading into 2019, with two new government-directed Internet disruptions observed in Africa, alongside disruptions caused by fiber cuts and other network issues that impacted a number of countries around the world.
Initially covered in last month’s overview, the Internet disruption in the Democratic Republic of Congo continued into January, lasting through the third week of the month. Government authorities reportedly cut off Internet access in the country in December to prevent “rumor mongering” in the run-up to presidential elections.
An attempted military coup in Gabon led to a day-long Internet disruption in the country. The disruption started just after 07:00 UTC on January 7, as seen in the figure below, which shows clear declines in the Traceroute Completion Ratio and BGP Routes metrics, as well as a disruption to the usual diurnal pattern seen in the DNS Query Rate metric. Although the coup was reportedly thwarted hours later, Internet connectivity was disrupted until around 11:00 UTC on February 8.
The disruption is also clearly evident in the Traffic Shifts graphs for AS16058 (Gabon Telecom), shown in the figure below. During the time that the Internet in Gabon was shut down, the number of successful traceroutes from Internet Intelligence measurement agents to endpoints in the network fell to near zero.
Just a week later, protests over fuel prices in Zimbabwe led the country’s government to order a social media ban, leading telecommunications service providers including Telone and Econet to block access to applications including Twitter, Facebook, and WhatsApp. (Telone is a state-owned operator and Zimbabwe’s largest fixed Internet provider, and Econet (Liquid Telecom) is Zimbabwe’s largest mobile operator.) Shortly thereafter, these providers moved to block Internet access entirely, leading to the disruption on January 15 & 16 seen in the figure below.
The impact of these shutdowns can be seen in the Traffic Shifts graphs for the respective network providers shown in the figures below, with the number of completed traceroutes to endpoints in each network dropping significantly over January 15 & 16.
The graphs above also show another disruption occurring on January 17 & 18, lasting for approximately 18 hours. This resulted from a directive issued by the Zimbabwean government ordering another shutdown, despite a pending court action challenging the legality of the earlier shutdown. A High Court justice ultimately ruled that the government directives ordering Internet shutdowns were illegal.
In November 2017, an Internet Exchange Point (IXP) was launched in Harare, Zimbabwe, providing a venue for local network providers to come together to exchange traffic. The figure below shows incoming and outgoing traffic levels at the Harare IXP during both periods of disruption – traffic in both directions drops to effectively zero.
On January 4, a reported fiber cut near Norilsk, Russia forced two local network providers onto backup satellite connections, significantly impacting performance. As shown in the Traffic Shifts graphs below for AS29520 (Rosintel) and AS58037 (Masterra), traceroutes targeting endpoints within the networks shifted to satellite connectivity providers with little observed disruption to service, but at the cost of much higher latency. A Facebook post on January 7 from the fiber operator reported that the damage to the fiber had been fixed, and that services had been restored.
It appears that a similar issue may have occurred at the end of January as well, forcing Masterra onto a higher latency satellite connection again for about a day and a half, as shown in the Traffic Shifts graphs below. A Facebook post from the aforementioned fiber operator references a fiber break that occurred on January 28, citing an expected 72-hour repair time, but it is not clear if that is related to this disruption.
Brief Internet disruptions in Greenland were observed on January 5 & 8, as shown in the figure below. All three measured metrics were impacted for between 90-120 minutes on both days. The disruptions may have been related to work being done to repair a reported break in a submarine cable connecting Greenland to Iceland – the break happened close to Qaqortoq, Greenland on December 28, 2018.
Tele Greenland is the largest telecommunications company in Greenland, and the impact of the submarine cable break on AS8818 (Tele Greenland) can be seen in the Traffic Shifts graphs below. During the two observed disruptions, the number of completed traceroutes to endpoints in the network dropped precipitously, while those that did complete successfully saw increased latency.
At approximately 07:30 GMT on January 20, a reported failure in the Tonga Cable disrupted Internet connectivity to Tonga. The Tonga Cable connects the island nation to Fiji, where Internet traffic can take advantage of Fiji’s connection to the Southern Cross Cable Network, connecting to Australia, New Zealand, and the United States. The cable’s failure resulted in a significant decrease in the number of completed traceroutes to endpoints in the country as well as a visible decline in the number of routed networks in the country, and the DNS Query Rate metric dropped to zero at times.
The figure below shows that the metrics gradually recovered over the next several days as Tonga’s Internet connectivity shifted to satellite backup. On January 28, a cable repair ship arrived in Tonga, and a day later, had located the damaged submarine cable, but found it around 100 meters south-east from where it was originally laid, and subsequently identified a second fault in the cable.
Other Network Issues
During the last few days of December 2018, and continuing into January 2019, network providers in South Sudan reportedly had to revert to backup geostationary satellite connectivity when microwave links to Uganda went down. As shown in the Traffic Shifts graphs below, the associated latency for traceroutes targeting endpoints in these networks increased by nearly 2.5x on December 28 & 29, December 31, and January 2 & 3 as they shifted to SES Astra satellite links.
A significant disruption in Internet connectivity was observed in the Country Statistics graphs for Gambia on January 16, as shown in the figure below. The next day, in response to user complaints regarding slow service, local provider Gamtel posted a Tweet (embedded below) explaining that planned maintenance by Portugal Telecom between January 16-20 could impact connection speeds.
The impact of this scheduled maintenance can be seen in the figures below, which show Traffic Shifts graphs for AS25250 (Gamtel) and AS37503 (Unique Solutions). In the graphs for Gamtel, the maintenance being performed by Portugal Telecom apparently caused a multi-hour drop in the number of completed traceroutes to endpoints within the network, along with a significant increase in associated latency. The Unique Solutions graphs show that traceroutes to endpoints within the network failed over from Portugal Telecom to an alternate provider, with only a brief disruption to connectivity, and a slight decrease in associated latency.
In addition to the disruptions reviewed above, an issue was observed in Haiti, where a unknown problem on January 5 at Digicel Haiti disrupted connectivity in the country for approximately five hours. Local providers in the Cook Islands and South Sudan apparently experienced problems with satellite connectivity to O3b on January 24 and January 27-29 respectively, resulting in multi-hour disruptions to Internet connectivity within these countries.
Problems with submarine cables, terrestrial fiber, and satellite/microwave links have long been threats to Internet connectivity, and it was almost expected that we’d see these occur soon after 2019 began. While arguably avoidable, their impact can generally be mitigated through disaster recovery plans that include backup connectivity through alternate providers and alternative types of connections. The new year also kicked off with two significant government-directed Internet disruptions in Africa, along with one that began in December and continued into 2019. Unfortunately, providers have minimal, if any, recourse when these types of shutdowns are ordered.