Issues at Facebook – Robert Hill, CEO

Yesterday’s Facebook outage caught a lot of people off guard and created a great deal of speculation. Facebook has gone, in the space of ~17 years, from a narrow scope social tool built in a dorm room to a monster tech giant with its tentacles into many aspects of everyday life. Facebook has become an international commerce, communication and news tool, and this platform literally, and almost unbelievably, disrupted the lives of many people, some of which seemingly live their entire lives dependent upon the platform. Whole companies are run on it, marketplaces built, business transacted. There are unfortunately companies whose businesses halted completely because of the Facebook outages (which included FB companies such as Instagram and WhatsApp).

Santosh Janardhan (Facebook’s VP of Infrastructure) posted a blog entry about its origin, along with an apology for the “inconvenience caused by today’s outage across our platform”. The post notes that the outage was caused by “changes on the backbone routers that coordinate network traffic between our datacenters”, basically a botched internal update / configuration change. This was most likely a Border Gateway Protocol (BGP) technical issue. The most famous example of this, until yesterday, was in 2008 when the Pakistan Telecommunication Authority (PTA) made a decision to block YouTube traffic to and from their country. As an AS (autonomous system) the PTA incorrectly formatted the update. This led to rapid global propagation that resulted in a majority of global YouTube traffic being incorrectly routed to them thereby overloading their systems and effectively bringing YouTube down. In all actuality, YouTube servers themselves were not actually down, the traffic was just not being routed to them- so they were “down” or inaccessible. China, Russia, and Iran have all had their own instances of such global traffic rerouting, but today’s Facebook outage was orders of magnitude bigger.

The question that should be on everyone’s mind is the unfortunate timing and the extraordinary length of time to restore traffic. What are the dangers of a more insidious possibility, an actual BGP hack? Originally there were very few AS, but now there are estimated to be 80,000. While this has provided some built-in redundancy, it also has created some unintended vulnerabilities. What if a nation-state or simply a disgruntled engineer decided to introduce a virtual detour sign on the internet superhighways simply rerouting the traffic to a black hole? A malicious attacker does not need to take down a well-protected server farm if they can simply prevent traffic from reaching the desired destination. What about other sites we have come to rely on with the work from home paradigm shift? Many people rely on Grub-Hub and Uber Eats for their meals, banking sites for transacting financial business, and sites like Amazon for necessities of daily life…all without leaving the “safety” of their homes. This could very well be the harbinger of a new set of threats and attacks against business and individuals as well, or it could simply be a Facebook engineer or contractor having a bad day. Having worked with DNS (Domain Name Services) for many years, I know firsthand how frustrating DNS entry issues can be, and how easy it is to fat finger an IP address for a server or gateway, or mess up a configuration setting.

Either way, yesterday’s events bring business owners and leaders an opportunity to make sure that we are thinking about risks to our companies. This is a call to action on being proactive and prepared. To getting a handle on the assets we own, where our systems and processes are vulnerable, being risk aware and knowing what our alternatives are if one of our critical systems go down.

Our mission is to help organizations identify risks, prioritize them as they apply to their business, and manage the remediation process. Reach out to me or one of my team if you’d like to have an executive conversation about how we have done that in our business at Cyturus and helped others to do the same.

Why Cybersecurity – written by Jeff Ellis, VP Marketing

The Information Age created an insatiable desire to make decisions based on facts and data, rather than intuition, whether it be credit decisions, medical decisions, industrial decisions or other types of behavioral decisions. All of this lead to a creation of vast amounts of data being collected on any number of things, events or situations. As data becomes more readily available and our ability to process that data has become more efficient through the use of SaaS programs, decision engines, scoring algorithms and the like, we as consumers have also come to expect a greater level of speed and efficiency as it relates to access. Access to what?, you might ask; quite simply the answer is access to everything. We want quick access to credit application decisions, quick access to our bank account information, quick access to the internet, quick access to our doctors, quick access to service providers ect. And thus we created things like; online banking, patient portals and service apps which has opened the door for an “always on” environment. While this phenomenon has certainly solved many problems it has also created huge problems too.

Cybersecurity is one problem in particular that is ever increasing and evolving and all too often individuals and organizations alike are unaware. Cyber criminals are constantly attempting to breach the vast amounts of data that has been collected and continues to be collected and these attempts are coming at an alarming rate. According to a recent article in techjury.net here are “The Most Telling Cyber Security Statistics in 2020”:

  • It takes half a year to detect a data breach.
  • 43% of all cyber attacks are aimed at small businesses.
  • 91% of attacks launch with a phishing email.
  • A business falls victim to a ransomware attack every 14 seconds.
  • 38% of malicious attachments are masked as one Microsoft Office type of file or another.
  • Cyber criminals managed to exploit the credit cards of 48% of Americans back in 2016.
  • The global cost of online crime is expected to reach $6 trillion by 2021.

If as a society and a business community we are not ready to defend ourselves then we could be in real trouble. And, this is not a problem that can be placed at the feet of IT professional alone to solve. This issue must be attacked through a culture of cybersecurity understanding and awareness of not only how these attacks happen but why. As business organizations we need to be looking at our entire enterprises to understand where our exposures are from personnel, to networks, to third parties, to end points and everything in between. And as individuals we need to be diligent in our behaviors and expectations of operating our daily lives in a manner that is safe and secure.

Be safe out there!