Surviving the data deluge

the volume of information they need to manage to stay on top of IP and brand protection issues is exploding.

A practical guide for IP and brand protection teams

We’ve heard it from teams of every size, at companies with the fastest-growing new products to brands that have been around for a century - the volume of information they need to manage to stay on top of IP and brand protection issues is exploding. This guide provides a practical approach to harness this data deluge and get ahead of the criminals.

Why it’s getting worse so quickly

As in all areas of society, the data generated by criminal activity is growing exponentially. With more of our daily lives revolving around digital devices and online services, there are even more opportunities for criminals to leverage. Every new service that comes out is another possible platform for criminals to exploit, and the tools and knowledge to increase their malicious activities are widely available on the darknet. Combined with other trends like the incredible growth in parcel shipping and you have a recipe for a data deluge.

What will happen next

You don’t need a crystal ball to see what is ahead - the rate of change is ever increasing, and there is plenty of new technology on the horizon that will continue to drive the growth. Teams we started working with 2-3 years ago were already groaning under the load yet they have seen a doubling or more of their workload in that timeframe. Imagine handling twice as much work as you do today within a few short years!

What you need to do

To survive the data deluge, the first step is to acknowledge that just like the criminals, you need to evolve and adapt. All the data that has been piling up around you already contains everything you need to identify the worst offenders and really get on top of the problem. What you need is a strategy to harness the data and rewire your brand protection team around the data-driven philosophy that is already driving many other teams with fast-growing brands. This whitepaper provides an outline.

  1. Build a business case We are constantly surprised by the number of teams we work with who forget this step. To get your management on board with the need for change (and the budget to achieve it), spend the time to build a business case to invest in solutions to the data deluge. Practical tip: a productivity tool can often pay for itself, by focusing your team where they will have the greatest impact (e.g. avoiding wasted effort) and by allowing your team to handle a growing workload without additional headcount. For example, if the cost to manage criminal activity grows at a projected rate over a forecasted period (e.g. 2 years), and you can estimate how much productivity improvement (e.g. 25%) could be achieved, a portion of these savings could be used to pay for a solution.
  2. Gather requirements Requirements gathering is mostly about the art of listening. One challenge is getting time with busy people, so try to give key stakeholders enough notice and preferably get them somewhere out of their normal work environment so they can stop juggling long enough to reflect. Don’t forget to talk to other stakeholders like management or external partners; their requirements can really “throw a wrench” in your careful plans if you forgot to include them early on.
  3. Inventory your data sources The big data sources are usually pretty obvious - many teams have one or more “systems of record” that are used in different parts of the organization, such as case management or CRM systems that are used everywhere. But if you really look at where the majority of data used on a day-to-day basis is coming from, you will usually find a surprising result: Microsoft Excel! From customs seizures to takedowns to lists of possible targets, good old Excel spreadsheets are often a big part of your team’s data inventory. And finally, if you are paying third parties to help you (for example, online monitoring and takedown services), their terms of use usually allow you to download your data or connect directly to it. This can be a valuable source of extra data to augment what you already have.
  4. Build your data model A data model describes all the elements and types of data that you have access to. We recommend building a model that uses the same terms the business uses (e.g. is it a “matter”, an “investigation” or a “case”?) and trying to simplify as much as possible. Try to keep to only the most important data for the first iteration - after all, you will be back (see our final section).
  5. Clean up your data Ah, the stories we could tell you. Tales of mysterious Chinese addresses, misspelled names, mismatched date formats, embedded quotes and more. The bottom line is that your data is much less clean than you think, and you will likely need to invest at least 50% of your project time on data clean up. Fortunately, there are many services available that can help, from address standardization to telephone number validation and more.
  6. Model your business processes Now that you have a business-oriented data model and some nice, clean data, it’s time to figure out the details about how people will be using it. We have found that if you write down the key activities that people on your team perform, you can often help them most by producing dashboards that line up with those activities. There is also usually a “long tail” of infrequent tasks that are best addressed by including some self-service abilities in the software platform that you choose, so that people can get access to the data they need.
  7. Train the people In data-oriented projects, the people leading the project are often the most qualified power users of the tools they select. This can lead them to believe that everyone will find the tools easy to use and will get the benefits straight away. In reality, every organization has people with a wide range of interest and capability, and you will need to invest energy in training. We find it is usually best for the focus of the training to be on the particular data and processes the team uses, rather than generic training on the technology itself.
  8. Repeat If you think you are done when your project is complete and your people are trained, ask yourself: are the criminals going to continue doing what they did yesterday? The challenge with managing criminal activity is that there is always incentive for criminals to find new and better ways to outsmart their competition (you!). We recommend an iterative approach where you pursue short versions of the cycle described above on a permanent basis.


If that sounds like a lot of complexity to manage, the good news is there is help available. Hubstream has a next-generation software platform that can serve all the requirements outlined above called Hubstream Intelligence for Brand Protection that comes with a well-defined implementation plan to get you where you need to be.

About the author

John Hancock is a data analytics industry veteran, an author and speaker on the challenges posed by the intersection of data and crime. He is the CEO of Hubstream Inc, a Seattle-based cloud software company that helps the world’s biggest companies and most sophisticated law enforcement agencies solve crimes faster.