- How secure is hadoop ecosystem?
- How users, administrators and analysts will use big data in secure manner?
- How to fit in hadoop ecosystem into existing enterprise security models?
Just open hadoop architecture document “assumptions and goals” here and you will notice security was never there in consideration. Hadoop was not built with enterprise security in mind. But when enterprises start adopting it, they will definitely ask questions similar to those I mentioned.
If you are using hadoop in a closed secure environment and no one except few trusted members are accessing it for performing some POCs, you may ignore security for a while.
But this is not how enterprises are adopting hadoop and building hadoop based systems. If you are interested in understanding more about how enterprise hadoop adoption is progressing, you may have to spend some time goggling around but for today’s discussion, I will refer a simplified version of high level architecture shown by HortonWorks.
If you are using hadoop in a closed secure environment and no one except few trusted members are accessing it for performing some POCs, you may ignore security for a while.
But this is not how enterprises are adopting hadoop and building hadoop based systems. If you are interested in understanding more about how enterprise hadoop adoption is progressing, you may have to spend some time goggling around but for today’s discussion, I will refer a simplified version of high level architecture shown by HortonWorks.
Just a quick scan of above diagram shows, your hadoop systems will be accessed by various applications and users in many ways across the globe over secured and unsecured networks. In any of such platforms, security is a serious concern but for hadoop, it is more vital.
Why? Why more vital for hadoop?
Just look back again at architecture diagram, data is flowing into hadoop from every possible sources, your CRM, ERP, Logs, Click stream, Sensors, Social media etc. This data is precious for enterprise and we must secure it from all possible security threats.
Why? Why more vital for hadoop?
Just look back again at architecture diagram, data is flowing into hadoop from every possible sources, your CRM, ERP, Logs, Click stream, Sensors, Social media etc. This data is precious for enterprise and we must secure it from all possible security threats.
Your big data platform doesn’t only store your big data but also all the insights, patterns and analytics results which you have derived or discovered from your big data. You can’t even ignore those intermediate results which you generated during the process of discovering insights from your big data.
Security is a big serious concern and an important aspect of big data technology which you have to take care. You will also have to manage risk associated with big data security.
Security is a big serious concern and an important aspect of big data technology which you have to take care. You will also have to manage risk associated with big data security.
Next question comes what are the aspects of security which we have to consider and cover for. To answer this question, we have to consider enterprise perspective of security.
What is an enterprise security?
Enterprise security is mainly driven by three things.
What is an enterprise security?
Enterprise security is mainly driven by three things.
- Legislation
- Internal policies
- Business drivers
Legislation forces some regulatory and standard compliance on various enterprises. It might be based on global regulations or may be local laws and regulatory needs. Few examples of such standards and regulations are given below.
Global Standards
ISO/IEC 27002:2005 – Code of Practice for Information Security Management
ISO/IEC 27001:2005 – Information Security Management System Requirements
ISO/IEC 15408 – Evaluation Criteria for IT Security
ISO/IEC 13335 – IT Security Management
PCI-DSS – Payment card industry – data security standards
COBIT – Control objectives for information and related technology
ITIL – ISO/IEC 20000 SERIES
ISO/IEC 27001:2005 – Information Security Management System Requirements
ISO/IEC 15408 – Evaluation Criteria for IT Security
ISO/IEC 13335 – IT Security Management
PCI-DSS – Payment card industry – data security standards
COBIT – Control objectives for information and related technology
ITIL – ISO/IEC 20000 SERIES
Regulation in US
SOX – Sarbanes-Oxley Act of 2002
COSO – Committee Of Sponsoring Organizations of the Tread way Commission
HIPPA – Health Insurance Portability And Accountability Act 1996
FISMA – Federal Information Security Management Act
FIPS – Federal Information Processing Standards
COSO – Committee Of Sponsoring Organizations of the Tread way Commission
HIPPA – Health Insurance Portability And Accountability Act 1996
FISMA – Federal Information Security Management Act
FIPS – Federal Information Processing Standards
Regulations in EU
Data Protection Act 1984 amended 1998 – UK
Data Protection Act 2004 – France
Directive 95/46/EC of the European Parliament and of the Council – 1995 – EU
RIP/RIPA – Regulation of Investigatory Powers Act 2000 – EU
Federal Data Protection Act 2006 – Germany
Data Protection Act 2004 – France
Directive 95/46/EC of the European Parliament and of the Council – 1995 – EU
RIP/RIPA – Regulation of Investigatory Powers Act 2000 – EU
Federal Data Protection Act 2006 – Germany
Internal policies and business drivers are specific to enterprise and vary from industry to industry.
Based on all this discussion and in-depth analysis of various needs I am trying to build a broader view of enterprise security using below diagram.
Above diagram clearly lists 8 key verticals of enterprise security and any big data solution will have to address all of these verticals to be adopted by any enterprise.
