The data challenge. What it is and how to avoid it.

The data challenge. What it is and how to avoid it.

Most companies, especially large ones, as well as governments departments, struggle with the amount of data they have accumulated over the years, either for themselves or for their customers. Most documents fall into two distinct categories, records and unstructured data. Typically, the challenges they are confronted with are as follows:

·         How much data is there across multiple platforms?

·         How many duplicates are there across multiple platforms?

·         Are duplicate, having the same revisions, have the same content (we should not assume that this is the case)?

·         Is the format readable (old format may not be read by today’s applications? (e.g. Microsoft Project Plan old files (over 10 years old files cannot be read with today’s application version)

·         Who owns the data if anybody?

·         Is there any retention associated with the data?

·         Are there any policies in procedures that define?

o   Ownership

o   Retention management

o   Backups and disaster recovery

o   Document Control and Management

·         Is the data available on paper, electronic media or both?

A few organisations will have a better control over records than they have with unstructured data. This is a shame as the bulk of their data will mostly be unstructured.

So, the question is why large companies can fall into this kind of mess? The answers are straight forward, if not embarrassing, as the roots of the issues are:

·         Sheer volume of information

·         Speed of accumulating new data

·         Lack of effective information management

This lack of proper management of data leads to an unpleasant reality that may or may not be properly accessed by management. Potential risks for the organisations are:

·         Driving up the costs of storage

·         Raising the cost of regulatory governance

·         Failing at meeting international Standards (e.g. ISO 9001)

·         Exposing the company to fines and even in some extreme case to legal challenges that could lead to imprisonments

·         Negative impact of employees’ productivity

·         Exposing the company to loss of control of the data (losing hard drives, memory sticks, hacking, etc…)

Although the data challenge generally impacts large companies, small and medium size ones should be aware of what are the steps not just to remedy a badly managed situation, but to make sure that they do not fall into the same situation when they start growing.

The first step is to take a good look at how ineffective the company-wide information governance is. Once the problems have been found, then a strategy must be generated, approved and funded to address each one of them. Such strategy must focus on the creation or update of policies, procedures and guidelines.

The second step, creating a strategy must be all inclusive as no data should be left in the dark.

The third step is to get the strategy approved and funded, thus leading to the creation of a project that will deliver the cure by delivering sound document control policies and procedures, centralising data, purging out what is not required any longer, defining ownership, applying retention management and moving over to automating data transactions (creation, reviews, approvals, archiving and disposal).

The fourth stage is about ensuring that training of the employees to the new policies and procedures, to the new ways of working take place and that a new company culture grows from it.

Finally, the final stage is to keep monitoring, via audits or automated processes, that failures are taking place.

Some of the topics illustrated in this blog, will be covered in more details in the future.