View Sidebar

Archive for category: Isilon

Unabated Experimentation is Way Forward in Big Data

Unabated Experimentation is Way Forward in Big Data

Big Data Experimentation

While it is true that analytical modeling is calling for nonstop testing of big data, the equation isn’t that straightforward and holds certain potential challenges.

The need of the hour is active experimentation in the big-data zone to help in-progress analytical model to make precise correlations. But since statistical models have their own risks, their astute application is going to be a must, especially as long as we want the results to be positive.

While a few groups are still hesitant, most full-size organizations have been able to hone their insight to realize that big data calls for incessant experimentation, and are all in support for the alteration. They also know, at the same time, that practical scenario of the booming field of big data involves certain risks associated with statistical models, especially when their implementation is not flawless.

Statistical Modeling –Practicality and Risks

Statistical models are simplified tools employed by data science to recognize and validate all major correlative aspects at work in a particular field. They can, however, make data scientists have a fake sense of validation at times.

And despite fitting the observational data quite rightly, various such models have been found to miss the real major causative factors in action. This is why predictive validity is often missing in the delusion of insight offered by such a model!

What May go Wrong?

Even though the application of a statistical model is practical in business, there is always a need to scrutinize the true, fundamental causative factors.

The lack of confidence may prove to be the biggest risk, particularly when you doubt the relevancy of the standard (past) correlations constituting your statistical model in near future. And obviously, predictive model of product demand and customer response in a particular zone which you have low confidence in will never be able to pull in huge investments during a product launch!

What is the Scope?

Even though there are certain risks involved, statistical modeling can never be completely dead. To be able to detect causative factors more quickly and effectively, statistical modeling will need to be based on real-world experimentation. This innovative approach that employs a boundless series of real-world experiments will be highly helpful in making big data business model and economy more authentic and reliable.

So How’s Real-world Experimentation Going to Be Possible? 

Exactly the way data scientists have developed advanced operational functions for ceaseless experimentation, big organizations look forward to encouraging their expert business executives to lead the charge in terms of running nonstop experiments and for better output. And to add to their convenience, the big data revolution has already offered in-database platforms for proper execution of a model and economical yet high-output computing power to make real-world experimentation feasible everywhere including scientific and business domains.

The basic idea is to prefer spending time, capital and other resources to conduct more low-risk experiments to putting extra efforts building the same models back and back again!

Monetizing Big Data: What 2014 Might Have in Store

Monetizing Big Data: What 2014 Might Have in Store

Once we are able to invest in the big data technology after successfully analyzing it, the next move will be to monetize it to obtain its monetary equivalent. To know what is the scope of big data monetization on 2014 and beyond, read on!

‘Big Data’ is already a familiar term for most of us, especially those who are into some serious business. It has been a hot topic in the media almost throughout the year 2013.

Big Data - Return on Investment - What 2014 Has in Store

All small and big businesses, however, are still trying to augment their knowledge about what actually big data is and what they should be doing about it and how. And what seems to be adding to the complications are the challenges involved in the process of big data investment.

Majorly, businesses don’t know how to obtain value from data and have to go a long way to be able to define the much-awaited big data policy. Even more importantly, they’ll have to attain the required skills and then execute them in a nifty manner to make the most of the strategies they’re working on!

Big Data – Future and Monetary Equivalent

While we are already in the first phase of the grand big data revolution where we’ve seen big investments in the technology, the next important step would be to generate revenue through big data.

Having a lot in reserve, the year 2014 is ready to play an important role in this regard:

Revenue Generation

Though businesses are all for huge investments in big data, they still need to predict how quickly it can generate revenue. The need of an effective way to measure ROI over a specific period of time may prove to be one of the potential challenges!

But despite all these assessments, most business leaders are expecting big data to be highly helpful in making the right business decisions. However, they believe that it won’t be possible to predict time and money associated with a ROI target without a guiding hand. This may cause giant businesses to opt for big data-based solutions rather than directly using big data as the only solution in 2014. The ultimate goal would be to boost up overall revenue by saving on costly technologies and data consultants.

Big Data as a Marketing Investment 

While it is true that big data has been more of a technology investment till now, we’ll see it as a marketing investment in 2014 and further, and retail brands will lead the charge in that case.

The key will be to persuade people to ‘buy’ by making all the offers directly customer-oriented. Big companies have already begun to prepare for the shift by motivating their CMOs, technology officers and information executives to work in unison to derive the best results.

Utilization of Big Data-based Solutions

With big data-based solutions surfacing quickly, all businesses will have to go for data analytics sooner or later. Though Google analytics have already been used for the same purpose for years, the latest big data-based solutions will allow all small and big companies to access solutions and methods that can ‘practically improve revenue.’ Hopefully, the year 2014 will be big for both those starting-up and well-established businesses in terms of using big data to get the best results!

Be Smart With Big Data

Be Smart With Big Data

smart dataSome companies get scared of big data. They think that since data is inherently dumb, a lot of it would be dumber still. But by being smart about big data, analysts can make sure that they get the most out of it. Handling big data can be a security risk and needs to be handled smartly.

The Present Way of Doing Things

Usually companies have one of three ways to handle data. They either go with the Heroic Model in which individuals take charge of requests and make decisions on their own without consulting with others. This model can work well for small businesses where individuals are usually aware of most situations across all areas of the business. But in bigger businesses, it can lead to confusion and chaos.

The Culture of Discipline on the other hand is one where individuals don’t make any decisions and follow a set of rules set by the management. Employees in this model can’t use data for their own decision making and just have to follow the processes set up for them.

The best way to handle data is to have a Data Smart Model in which data is managed on an evidence based management system. It is a combination of the first two methods and it works on a disciplined processing method but decision making is allowed at the individual level. This is the method that should be used to handle big data and it can result in smooth operation without much hassles.

How to Cultivate the Data Smart Culture

Certain steps need to be taken to create the data smart culture.

  • There should be a single source of truth. Decision making can be moved to the employee level but the guiding principles should be set from a single source.
  • Use ways to keep track of progress. Using a scorecard system, even on a daily basis, can help managers across different branches know how they are performing in relation to the other departments and they can then send in better data to record their progress.
  • Rules are important but there should be enough flexibility. Rules and guiding principles are needed but there should be flexibility to know when to bend the rules and when to break them. Sometimes what works in most parts of the country might not be best for a certain area. Businesses need to be able to adapt to such situations and change their rules accordingly.
  • Work on cultivating human resources. The people are the biggest asset of a company and it is important to educate them and provide them with the proper know-how to handle data. Managers need to be trained to educate the people working under them and give them a one to one engagement.

These steps can help businesses handle big data smartly and without much confusion. Every level needs to be trained to handle big data as the future is going to be all about big data.

Is Big Data a Threat to Your Privacy?

Is Big Data a Threat to Your Privacy?

Big Data is growing bigger every day and along with it the concern over invasion of privacy is also growing. Tracking all the data generated by your mobile and other devices and your interactions on social media, is beneficial for advertisers to tailor their ads to suit you. But there’s more to the story than that. Companies have now begun to come up with very creative ways to use real time data.

Let’s look at some interesting examples.

Smart Rubbish Bins in London

An advertising firm in London came up with the idea to use strategically placed dustbins to track the wifi signal of phones of the people passing by. They could use the serial number of the phones to track the movement of every individual. They could then use this data to show advertisements on the screen of these bins, that are targeted at the person passing by.

smartbins

Now even dustbins are becoming smart!

The officials have asked Renew, the responsible ad firm, to take down the smart dustbins as there has been a lot of concern about the invasion of privacy of the people.

Police Cars in Australia get Number Plate Recognition Cameras

The Aussies have come up with another great use of Big Data by using number plate recognition cameras that can read multiple number plates simultaneously and also search their database to find out all the information about that driver. They can tell if a car is stolen or if you have unpaid parking tickets just by looking at your car’s number plate.

police car

The hand of the law gets longer.

Are Such Examples a Threat to Your Privacy?

When CCTV cameras first came on the scene, the public responded to them with an outrage similar to what we see now in terms of Big Data. But once people got used to the new technology and saw the benefits in solving crimes and catching miscreants swiftly, the fears of Big Brother always watching them subsided.

The truth is that people will allow collection of any data as long as it is collected with their permission and it is used to create value for them. Instead of shoving ads in people’s faces, companies should try to find other ways to use Big Data, not only to reduce costs for the company but also to provide quality to the customer.

One great example to highlight the creative use of Big Data is the potential for insurance companies. Today all natural or man made calamities generate a lot of data in the social media.

data

Data about Hurricane Sandy

Insurance companies can use this data along with before and after images on Google Maps Street View, Flickr, Instagram etc. to find out how much destruction of property their clients have suffered.

torn houseThey can estimate the number and amount of claims that they will have to deal with. They can provide quick claim settlements to their customers which will be appreciated by all and people will readily agree to data collection if they are told of such rewards.

Great Opportunities

A Westpac survey showed that it only took 30 months for mobile usage to reach 1 Million as compared to 80 months it took for online usage to reach the 1 Million mark.

graphThis means that there are great opportunities available to use this rapidly growing Big Data but it will have to be done with care and while keeping the interests of the consumer in mind.

Enterprise Ready Hadoop Infrastructure from EMC – Isilon

Enterprise Ready Hadoop Infrastructure from EMC – Isilon

With increased reliance on technology and large scale usage of applications and IT systems, the amount of structured & unstructured data stored and processed by a typical modern-day enterprise has been growing very rapidly. Organizations today, lest they’re okay with the idea of being left-behind in the race, require highly efficient, effective and scalable storage solutions to manage this growth.

Modern day organizations require high-end storage systems also because the latter helps provide powerful analytics; they can draw information of concern from data. EMC Isilon scale-out Network-attached storage (NAS) with native Hadoop Distributed File System (HDFS) provides Hadoop users access to shared storage infrastructure that helps minimize the void between Big Data Hadoop and IT analytics.

The lsilon NAS integrated with HDFS offers customers a solution to accelerate enterprise ready development of Apache Hadoop. Until now, customers of Hadoop have benefited from storage infrastructure solutions that weren’t really optimized for big data storage, thus limiting the scope of Hadoop’s applicability in large enterprises. But, EMC Isilon with native HDFS tackles this challenge well and offers an all-inclusive enterprise ready storage system to collect, protect, analyze and share data in Hadoop environment.

Enterprise Ready Hadoop Infrastructure from EMC - Isilon

By integrating Hadoop natively in an enterprise-class storage solution, Isilon has enabled customers to benefit from a comprehensive data protection system (irrespective of the size of the Hadoop data). By combining EMC Isilon scale-out NAS with native HDFS, EMC will be able to reduce the complications related to Hadoop usage to allow enterprises to extract valuable data from the gigantic heaps of unstructured & structured data.

EMC Isilon provides Hadoop customers a built-in entrance to enterprise data protection; this is made possible with the integration of Isilon scale-out NAS storage system and native HDFS. This integration of Isilon and HDFS eliminates any one point failure with open source Apache Hadoop that enterprises are using; further, the combination allows customers to use a Hadoop system of choice to accelerate their Hadoop adaptation in enterprise ready environment.

Industry’s first scale-out storage system with native HDFS offers the following advantages: 

  • Enterprises can utilize more benefits of Hadoop
  • Reduces risks
  • Increases organization knowledge

The reason why enterprises need to consider ‘HDFS plus Isilon’ is that there’s no ingest necessary anymore. It’s comparatively cheaper and still, the performance is better. With multiple enterprise-features, multi-protocol access and Hadoop multi-tenancy, ‘HDFS on Isilon’ supports nearly everything you’d possible want to work with such as Pivotal, Apache, Cloudera and Hortonworks. NameNode SPOF and 3x Mirroring, two key challenges with DAS Hadoop are eliminated too!

Advantages of EMC Isilon storage implementation over traditional implementation

  • It offers scale-out storage to facilitate multiple workflow and applications
  • No downtime associated, it is distributed in NameNode
  • Provides matchless storage efficiency
  • Offers independent scalability to compute and store separately
  • Provides end-to-end data protection using SnapshotIQ, SynclQ and NDMP Backup

Benefits an enterprise derives from data storage & analytics solution – Hadoop

Hadoop as an enterprise ready big data analytics solution can help store, analyze, structure and visualize big amounts of structured & unstructured data. Hadoop is especially beneficial because it enables users to process unstructured big data, to give it structure so that it can be used for the advantage of the enterprise.

a)   Benefits an enterprise derives

  • Enhanced business agility
  • Easier data management
  • Faster and more convenient data analytics
  • Reduction in time and cost of infrastructure and maintenance
  • Ability to accommodate and analyze irrespective of type or size

b)   Hadoop enterprise ready EMC Isilon advantages:

  • Dependable security
  • Scalable storage solution
  • Continuous availability
  • Existing infrastructure and simple integration
  • Easy deployment and faster administration

EMC Hadoop Starter Kit (HSK)

For extracting insights on customer sentiments and other such information from big data, you will need the Hadoop integration if you are an enterprise that uses VMware Vsphere and/or EMC Isilon . Hadoop with Isilon integration becomes enterprise-ready and helps your data architecture deal with new opportunities provided by data most diligently along with the existing tasks.

Now, to make things even simpler for an organization that uses VMware Vsphere and EMC Isilon, an EMC Hadoop Starter Kit has been developed (video). This HSK step-by-step guide is designed to help enterprises learn and discover the all encompassing potentials of Hadoop.

VMware has also started an open source project (called Serengeti) that can help automate the management and deployment of Hadoop clusters on vSphere. With a virtualized infrastructure, Hadoop can be run as a service.

Whether you are a seasoned Hadoop user or a newbie, all can equally benefit with the HSK because of following reasons:

Rapid provisioning: Most of the Hadoop cluster development can be automated with expertise. Thus, the guide takes you through the process of creation of Hadoop nodes and to set up and start Hadoop service on a cluster, which makes it ever so simple for you to execute.

High availability: High availability protection with use of virtualization platform ensures that single point of failure in Hadoop storage solution can be protected.

Profitability: Enterprises can use and benefit from any Hadoop distribution within the big data application lifecycle; this, with zero data migration.

Elasticity: The same physical infrastructure can be shared amid Hadoop and other application, since, the Hadoop capacity can be scaled to and fro according to demand.

Multi tenancy: Hadoop infrastructure offers multi tenancy option, which means different tenants can have virtual machines provided to them, thus enhancing data security.

EMC Hadoop Starter Kit combines the benefits of VMware vSphere with Isilon scale-out NAS in order to help achieve big data storage goals and added analytics solution.

Some of the reasons why the HSK can be considered as the outright solution have been mentioned above. The merits, especially ‘profitability,’ explains that users can use Hadoop distribution all through the big data application lifecycle with zero data migration that includes, Hortonworks, Pivotal HD, Cloudera and Apache Open Source etc.

This means that starting Hadoop project with EMC Isilon scale-out NAS, enterprises can profit with zero data migration when they have to move from one Hadoop distribution to another. This implies that user can run multiple Hadoop distributions for same data without data duplication.

EMC Isilon’s Notable Collaborations

In addition, Isilon also shares a good collaborative effort with companies like Splunk, Rackspace and Rainstor. EMC Isilon scale-out NAS is no doubt the finest storage system offering users an opportunity to scale capacity and performance of data to meet their needs. To benefit Hadoop users, Isilon has teamed up with Splunk, Rackspace and Rainstor for additional benefits.

Isilon and Splunk: Splunk for Isilon app integrates EMC scale-out NAS with Splunk. The team up of EMC and Splunk helps enterprises manage avalanche of data across virtual, cloud and physical environments to transform this data into real time insight for the user.

Isilon and Rackspace: EMC Isilon helps enterprises to store, consolidate, analyze and use data and applications exceeding 100 TB. Rackspace offers its services to EMC Islion NL400 and X400 high density and large capacity models to perform their tasks diligently for greater benefit of enterprises.

Isilon and RainStor: The combination of EMC and RainStor helps enterprises run the Hadoop distribution anywhere. The RainStor’s unique data compression technique helps enterprises to analyze their large data sets with more efficiency and greater predictability.