3 core principles for secure data integration

On the subject of knowledge, sharing just isn’t all the time caring.

Sure, the increased flow of data throughout departments like advertising and marketing, gross sales, and HR is doing a lot to energy higher decision-making, improve buyer expertise, and — finally — enhance enterprise outcomes. However this has severe implications for safety and compliance.

This text will focus on why, then current three core ideas for the secure integration of knowledge.

Democratizing entry to knowledge: An essential caveat 

Available on the market right this moment is an unbelievable vary of no-code and low-code tools for shifting, sharing and analyzing knowledge. Extract, rework, load (ETL) and extract, load, rework (ELT) platforms, iPaaS platforms, knowledge visualization apps, and databases as a service — all of those can be utilized comparatively simply by non-technical professionals with minimal oversight from directors.

Furthermore, the variety of SaaS apps that companies use right this moment is constantly growing, so the necessity for self-serve integrations will possible solely enhance.

Many such apps, equivalent to CRMs and EPRs, include delicate buyer knowledge, payroll knowledge, invoicing knowledge and so forth. These are inclined to have strictly managed entry ranges, so so long as the info stays inside them, there isn’t a lot of a safety danger. 

However, as soon as you’re taking knowledge out of those environments and feed them to downstream methods with fully totally different entry stage controls, there emerges what we will time period “entry management misalignment.” 

Individuals working with ERP knowledge in a warehouse, for instance, might not have the identical stage of confidence from firm administration as the unique ERP operators. So, by merely connecting an app to a knowledge warehouse — one thing that’s an increasing number of usually changing into needed — you run the danger of leaking delicate knowledge.

This can lead to violation of rules like GDPR in Europe or HIPAA within the U.S., in addition to necessities for knowledge safety certifications like SOC 2 Kind 2, to not point out stakeholder belief.

Three ideas for safe knowledge integration

Tips on how to forestall the pointless move of delicate knowledge to downstream methods? Tips on how to maintain it safe in case it does have to be shared? And in case of a possible safety incident, how to make sure that any injury is mitigated?

These questions will likely be addressed by the three ideas beneath.

Separate issues

By separating knowledge storage, processing and visualization features, companies can decrease the danger of knowledge breaches. Let’s illustrate how this works by instance.

Think about that you’re an ecommerce firm. Your most important manufacturing database — which is linked to your CRM, fee gateway and different apps — shops all of your stock, buyer, and order information. As your organization grows, you determine it’s time to rent your first knowledge scientist. Naturally, the very first thing they do is ask for entry to datasets with all of the abovementioned data in order that they will write knowledge fashions for, let’s say, how the climate impacts the ordering course of, or what the most well-liked merchandise is in a selected class.

However, it’s not very sensible to present the info scientist direct entry to your most important database. Even when they’ve one of the best of intentions, they could, for instance, export delicate buyer knowledge from that database to a dashboard that’s viewable by unauthorized customers. Moreover, working analytics queries on a manufacturing database can sluggish it all the way down to the purpose of inoperability.

The answer to this downside is to obviously outline what sort of knowledge must be analyzed and, through the use of numerous data replication techniques, to repeat knowledge right into a secondary warehouse designed particularly for analytics workloads equivalent to like Redshift, BigQuery or Snowflake.

On this manner, you forestall delicate knowledge from flowing downstream to the info scientist, and on the similar time give them a safe sandbox surroundings that’s fully separate out of your manufacturing database.

Use knowledge exclusion and knowledge masking methods

These two processes additionally assist separate issues as a result of they forestall the move of delicate data to downstream methods completely.

In actual fact, most knowledge safety and compliance points can truly be solved proper when the info is being extracted from apps. In spite of everything, if there isn’t any good motive to ship buyer phone numbers out of your CRM to your manufacturing database, why do it? 

The thought of knowledge exclusion is straightforward: If in case you have a system in place that means that you can choose subsets of knowledge for extraction like an ETL tool, you possibly can merely not choose the subsets that include delicate knowledge.

Bu, in fact, there are some conditions when delicate knowledge must be extracted and shared. That is the place data masking/hashing is available in.

Let’s say, as an example, that you simply need to calculate well being scores for purchasers and the one smart identifier is their e-mail tackle. This is able to require you to extract this data out of your CRM to your downstream methods. To maintain it safe from finish to finish, you possibly can masks or hash it upon extraction. This preserves the distinctiveness of the data, however makes the delicate data itself unreadable.

Each knowledge exclusion and knowledge masking/hashing will be achieved with an ETL instrument.

As a facet word, it’s value mentioning that ETL instruments are usually thought of safer than ELT instruments as a result of they permit knowledge to be masked or hashed earlier than they’re loaded into the goal system. For extra data, seek the advice of this detailed comparability of ETL and ELT tools.

Hold a robust system of auditing and logging in place

Lastly, be sure that there are methods in place that allow you to know who’s accessing knowledge and the way and the place the info is flowing.

After all, that is essential for compliance as a result of many rules require organizations to show that they’re monitoring entry to delicate knowledge. Nevertheless it’s additionally important for shortly detecting and reacting to any suspicious habits.

Auditing and logging is each the inner duty of the businesses themselves and the duty of the distributors of knowledge instruments, like pipelining options, knowledge warehouses and analytics platforms.

So, when evaluating such instruments for inclusion in your knowledge stack, it’s essential to concentrate to whether or not they have sound logging capabilities, role-based entry controls, and different safety mechanisms like multi-factor authentication (MFA). SOC 2 Kind 2 certification can also be a superb factor to search for as a result of it’s the usual for the way digital firms ought to deal with buyer knowledge.

This manner, if a possible safety incident ever does happen, it is possible for you to to conduct a forensic evaluation and mitigate the injury.

Entry vs. safety: Not a zero-sum recreation

As time goes on, companies will more and more be confronted with the necessity to share knowledge, in addition to the necessity to maintain it safe. Thankfully, assembly one in every of these wants doesn’t must imply neglecting the opposite.

The three ideas outlined above can underlie a safe knowledge integration technique in organizations of any measurement.

First, determine what knowledge will be shared after which copy it right into a safe sandbox surroundings.

Second, each time attainable, maintain delicate datasets in supply methods by excluding them from pipelines, and be sure you hash or masks any delicate knowledge that does have to be extracted.

Third, guarantee that your corporation itself and the instruments in your knowledge stack have robust methods of logging in place, in order that if something goes flawed, you possibly can decrease injury and examine correctly.

Petr Nemeth is the founder and CEO of Dataddo.