Big Data is omnipresent, yet relatively unknown. It is a technology that affects our everyday lives in so many ways, though most of us do not know it exists. In this blog I attempt to uncover a phenomenon that has become pervasive in many aspects of life and is now impacting, amongst others,  the waste, recycling and environmental sectors.

Before offering some insight into what Big Data is, let’s put forward a few examples that are easy to relate to: A 20’s something female updates her relationship status on Facebook from single to engaged.  She then begins to see subtle (and sometimes less than subtle) advertising for wedding dress shops, romantic holiday destinations and even hotels in her home vicinity that offer the full wedding package. Now, Facebook do not employ huge teams of data ninjas that monitor your every change in circumstance to then present you with appropriate advertisements, they employ Big Data. Recently, a piece of research from Cambridge University and Microsoft across 58,000 volunteers demonstrated a remarkable ability to predict a person’s age, gender, political affiliation and even sexual orientation just from their “Likes” on Facebook. Apparently clicking “Like” on subjects such as “Britney Spears” or “Desperate Housewives” were among the signs of homosexual orientation!

So what is Big Data?

Well, first of all, it is not just about the volume of data, though the volume of data that a business could not ordinarily manage is one aspect. We currently create 2.5 quintillion (that’s 18 zeros) bytes of data per day, 90% of which has been created in the last 2 years, so the amount of data we create is increasing exponentially. And where does all this data come from? Well, a huge amount does come from social media, but it also comes from non-human sources such as GPS devices, machine sensors and scientific sources. If we consider the electricity smart-grid planned for the UK by 2020, if every smart meter takes a reading every 30 minutes ( 17,500 per annum) across 53 million homes and businesses, this gives you a sense where the data explosion is coming from.

Another aspect, when defining Big Data, is the type of data. Traditional transactional databases have defined and fixed structures; field types and lengths, table relationships etc. But over 80% of data created around the globe is of a unstructured or semi-structured form, for example, this article is unstructured data, and twitter messages could be classed as semi-structured. We also have images (still or video) and sound coming into play as the technology advances as quickly as the data is produced.

The speed with which data is produced is a challenge, and technologies are continuing to develop to capture that data and, where relevant, turn it into something meaningful and insightful as quickly as the data is produced. It is only in the last decade that sat-nav has become common place. Approximately five years ago sat-nav services developed to include identification of traffic black-spots and average vehicle speeds, allowing you to re-route. These initial services were quite primitive and captured data from a relatively low number of fixed cameras on main routes. Today, other sources are used, including sharing of information with the mobile phone providers – all mobile phones give out a periodic signal and position and can therefore contribute to the collation of real-time vehicle speed data on many more roads than ever before. This data is presented at a huge velocity and must be filtered very effectively if it is to provide dynamic re-routing to you and me.

How will it be used?

Big Data is already being used by most of us, we just haven’t perhaps recognised it. If we take the sat-nav example above then it is quite easy to see how we can gain benefit from its use. But what if we could take it a stage further. Waste and recycling companies often carry huge overheads, particularly from the depots manned and the vehicles deployed. Using Big Data we are able to analyse the location of our depots, the customers we service and the routes we drive, taking into account accurate data from our internal systems but also that from other sources. From here we can remodel our whole business, and look at alternative solutions that provide the right service to our customers, but potentially on much lower overheads. Software solutions, and powerful servers, can take all of that data and apply millions of what-ifs to come up with suggested changes – perhaps merging territories, closing operations in certain areas, or investing in new facilities in others. And it’s not just about the financial overhead, we can apply the same logic to CO2 measures, allowing us to  consider the impact of building a new EfW close to the waste producer(s) compared with transporting the waste to an existing facility we have already invested in.

A segment of the above data can also be used for more dynamic overhead reduction, by integrating with the actual containers; we already have the in-bin sensors that can transmit a small amount of data to tell us how full a bin is, and from that we can predict the point at which it is most economical to empty, whilst avoiding any messy overflow.

A very different example would be around social media. Often we see cases of nimbyism in the industry; local opposition to a new treatment facility. Using Big Data technologies we are able to track Sentiment on social networking sites. This is quite simple, but very powerful. Feeding sentiment tracking into your business allows you to react quickly to situations, and then influence market opinions. During the consultation and bid process for a new waste treatment facility you are able to use Big Data to your advantage, and to ensure your PR messaging promotes the positive opinion and corrects the negative. To sample it, I recommend a look at www.sentiment140.com and search on a few brands such as Amazon, HMV or even your own company name.  Whilst this simple tool gives a good picture of your current sentiment score, the measure becomes more powerful as a continuous stream, tracking movements up and down, and implementing alert tools that can wake you in the middle of the night it your score drops below a certain parameter.

Following on from one of our initial examples, as more energy is produced from waste, and as more knowledge becomes available from patterns plotted from the smart-meter data, it is a natural next step that we use that data to manage the in-flow of material into EfW sites, using algorithms to make best decisions to extract the best monetary value from the calorific value.

To conclude, whilst most people have been impacted by Big Data, not many know it yet. Data is only of value to us if it allows us to make good decisions. Big Data allows us to make big decisions that can have a big financial and environmental impact. The only question left is do you want to be an Amazon or a HMV?