View Sidebar

Articles by: Kamal Thakur

Unabated Experimentation is Way Forward in Big Data

Unabated Experimentation is Way Forward in Big Data

Big Data Experimentation

While it is true that analytical modeling is calling for nonstop testing of big data, the equation isn’t that straightforward and holds certain potential challenges.

The need of the hour is active experimentation in the big-data zone to help in-progress analytical model to make precise correlations. But since statistical models have their own risks, their astute application is going to be a must, especially as long as we want the results to be positive.

While a few groups are still hesitant, most full-size organizations have been able to hone their insight to realize that big data calls for incessant experimentation, and are all in support for the alteration. They also know, at the same time, that practical scenario of the booming field of big data involves certain risks associated with statistical models, especially when their implementation is not flawless.

Statistical Modeling –Practicality and Risks

Statistical models are simplified tools employed by data science to recognize and validate all major correlative aspects at work in a particular field. They can, however, make data scientists have a fake sense of validation at times.

And despite fitting the observational data quite rightly, various such models have been found to miss the real major causative factors in action. This is why predictive validity is often missing in the delusion of insight offered by such a model!

What May go Wrong?

Even though the application of a statistical model is practical in business, there is always a need to scrutinize the true, fundamental causative factors.

The lack of confidence may prove to be the biggest risk, particularly when you doubt the relevancy of the standard (past) correlations constituting your statistical model in near future. And obviously, predictive model of product demand and customer response in a particular zone which you have low confidence in will never be able to pull in huge investments during a product launch!

What is the Scope?

Even though there are certain risks involved, statistical modeling can never be completely dead. To be able to detect causative factors more quickly and effectively, statistical modeling will need to be based on real-world experimentation. This innovative approach that employs a boundless series of real-world experiments will be highly helpful in making big data business model and economy more authentic and reliable.

So How’s Real-world Experimentation Going to Be Possible? 

Exactly the way data scientists have developed advanced operational functions for ceaseless experimentation, big organizations look forward to encouraging their expert business executives to lead the charge in terms of running nonstop experiments and for better output. And to add to their convenience, the big data revolution has already offered in-database platforms for proper execution of a model and economical yet high-output computing power to make real-world experimentation feasible everywhere including scientific and business domains.

The basic idea is to prefer spending time, capital and other resources to conduct more low-risk experiments to putting extra efforts building the same models back and back again!

Are Businesses Already Expecting Healthy Big Data ROI?

Are Businesses Already Expecting Healthy Big Data ROI?

Businesses in the UK use big data to mostly support their sales and marketing campaigns, reveals Big Data Survey 2013 carried out by MBN Recruitment Solutions.

The survey maintains that more than 80% of the total survey respondents look forward to harness and leverage their data to be able to generate new revenue.

At the same time, over 95% people agree that more revenue generation is going to be the only purpose of businesses using big data in near future!

Use of Big Data Till Now and In Future

The first annual survey by MBN also exposes that over 71% of the total respondents have been using data analytics to foresee all major functions and aspects of future businesses doings around the world.

Are Businesses Already Expecting Healthy Big Data ROI

MBN non-executive chairperson, Paul Forrest concludes that companies use big data as they grow larger and need to stay competitive against potential competitors.

Most survey respondents believe that right now there is too low ROI in leveraging big data. They, however, are expecting greater ROI prospects in future. Also, over 40% respondents think that the current initiatives will eventually fetch desired results for businesses. Forrest told that one of the biggest issues has been the importance of tools, but 72% respondents believe that tools are important only in the beginning and it is people who unlock the set value on a later stage.

Monetizing Big Data: What 2014 Might Have in Store

Monetizing Big Data: What 2014 Might Have in Store

Once we are able to invest in the big data technology after successfully analyzing it, the next move will be to monetize it to obtain its monetary equivalent. To know what is the scope of big data monetization on 2014 and beyond, read on!

‘Big Data’ is already a familiar term for most of us, especially those who are into some serious business. It has been a hot topic in the media almost throughout the year 2013.

Big Data - Return on Investment - What 2014 Has in Store

All small and big businesses, however, are still trying to augment their knowledge about what actually big data is and what they should be doing about it and how. And what seems to be adding to the complications are the challenges involved in the process of big data investment.

Majorly, businesses don’t know how to obtain value from data and have to go a long way to be able to define the much-awaited big data policy. Even more importantly, they’ll have to attain the required skills and then execute them in a nifty manner to make the most of the strategies they’re working on!

Big Data – Future and Monetary Equivalent

While we are already in the first phase of the grand big data revolution where we’ve seen big investments in the technology, the next important step would be to generate revenue through big data.

Having a lot in reserve, the year 2014 is ready to play an important role in this regard:

Revenue Generation

Though businesses are all for huge investments in big data, they still need to predict how quickly it can generate revenue. The need of an effective way to measure ROI over a specific period of time may prove to be one of the potential challenges!

But despite all these assessments, most business leaders are expecting big data to be highly helpful in making the right business decisions. However, they believe that it won’t be possible to predict time and money associated with a ROI target without a guiding hand. This may cause giant businesses to opt for big data-based solutions rather than directly using big data as the only solution in 2014. The ultimate goal would be to boost up overall revenue by saving on costly technologies and data consultants.

Big Data as a Marketing Investment 

While it is true that big data has been more of a technology investment till now, we’ll see it as a marketing investment in 2014 and further, and retail brands will lead the charge in that case.

The key will be to persuade people to ‘buy’ by making all the offers directly customer-oriented. Big companies have already begun to prepare for the shift by motivating their CMOs, technology officers and information executives to work in unison to derive the best results.

Utilization of Big Data-based Solutions

With big data-based solutions surfacing quickly, all businesses will have to go for data analytics sooner or later. Though Google analytics have already been used for the same purpose for years, the latest big data-based solutions will allow all small and big companies to access solutions and methods that can ‘practically improve revenue.’ Hopefully, the year 2014 will be big for both those starting-up and well-established businesses in terms of using big data to get the best results!

Oracle Launches 5th Gen Database Machine

Oracle Launches 5th Gen Database Machine

Oracle Exadata Database Machine X4, the 5th gen database machine form Oracle is a revolutionary step in the field of database management. Keep on reading to know what it has to offer!

Oracle Launches 5th Gen Database MachineOracle recently launched Exadata Database Machine X4. It has hi-tech hardware and software that can increase capacity, boost performance and maintain quality and efficiency of service for database operation.

The update focuses mainly on the optimization of Online Transaction Processing (OLTP). The machine is mainly aimed at providing businesses with a permanent solution for all major database challenges and has advantages like Data Warehousing and Database as a Service (DBaaS).

Oracle Exadata Database Machine X4 – Features

  • The machine is the 5th gen of Oracle Exadata that was launched in the year 2008.
  • It is a fifth gen machine featuring improvements that focus on improved performance as well as quality of service for OLTP, Database as a Service and Data Warehousing.
  • It uses high speed flash compression and larger physical flash in perfect combination to increase the capacity of flash memory that eventually accelerates the performance of OLTP-based work.
  • The latest Flash Caching algorithms help accelerate the performance of all workloads in Data Warehousing.
  • Many databases can be merged with help of the Database as a Service design because of extreme capacity and performance. This will help businesses improve on quickness and more importantly, reduce costs.
Hadoop Security: Present and Future

Hadoop Security: Present and Future

Where the current level of Hadoop system can be relied upon for data protection and processing, there is still a need to improve Hadoop security to ensure foolproof big data security for coming times. To stay updated about the scope of a secure Hadoop cluster today and in times to come, one needs to know a few important things about it.

Hadoop Security Present and FutureSecurity is the foremost agenda that represents almost all major requirements within an organization, especially when it is about tasks like big-data processing. Hadoop registered a remarkable progress in last couple of years and has successfully addressed the most common worries like authorization, authenticity and above all, data protection. With more security enhanced Handoop clusters in the pipeline, though using the systems are banking upon the safety of all vital data in the future also.

Hadoop currently is engaged at the cutting edge to provide secure support to countless financial service applications and big private healthcare projects that operate in a high security-sensitive environment. Recent upgrades of Hadoop systems meet the key requirements of organizations demanding some of the world’s toughest security norms. With all the tight security controls incorporated in Handoop, the final objective remains flexibility and smooth data processing for now and in the future.

Hadoop Security Controls Dec 2013

 Security Controls for Hadoop at Present

Securing a Handoop cluster presents certain both small and big, which includes its distributed nature that to a large extent is even responsible for its success. For securing a system, a layered approach is the best and distribution happens to be one of the most complex barriers to it.

Following are the major layers that are in place to secure a cluster:


It is responsible for verifying the identity of both a system and a user accessing it. Pseudo authentication and Kerberos are the two authentication modes Hadoop is providing. While the first takes care of the trust among users, the latter secures the overall Hadoop cluster.


Authorization represents access freedom for users and a system. Hadoop relies on resource-level access control, file permissions in HDFS and offers authorization and a service-level access control.


Accounting makes it possible to track resource use in a system. MapReduce and HDFS that are the parts of Apache Hadoop offer base audit support. Apache Oozie functions as a workflow engine and offers audit trail for all services.

Data Protection 

This takes care of privacy of information. HDP protects the data in motion and HDFS holds up encryption at operating-system levels.

Security Controls for Hadoop in Future

Newer innovations in Hadoop security are focusing mainly on making various security frameworks to work in collaboration so that they can be easily managed. Here’s what Hadoop security system is going to be big at:

Granular Authorization and Enhanced Authentication

Verification technique in most Hadoop modules is in the process of being improved. This is mainly developed and fortified mainly because most users are demanding security hardened authorization model. Token-based validation will soon replace Kerberos to enhance the authentication process.

Encryption Data Protection and Improved Accounting

A more advanced encryption algorithm is a must for most channels. The focus would be on better encryption, mostly through HBase, HDFS and Hive. Another important step is going to be high-tech audit record correlation for easier reporting. With this system, the auditor would be able to predict the sequence of Hadoop component operations without having to take help from any external tools.

Korea opens first big data analytics centre

Korea opens first big data analytics centre

To bring its technology sector at par with the global technology giants, South Korea has set up country’s first big data analytics center. The big data analytics centre will enable researchers and businesses to refine and analyze big data for their projects.

South Korea’s ministry of Science, ICT and Future Planning have worked in collaboration with National Information Society Agency (NIA) to open ‘Korea Big Data Centre’ (KBiG), the country’s first big data analytics centre in the NIA building.

Big_Data_center south koreaThe Ministry of Science informed that the Korea Big Data Centre is set up to promote data analytics in government and private sector with the intention to transform Korea into a data driven creative economy.

The new centre will allow all kinds of researchers, businesses and hospitals etc., to process and analyze the avalanche of data for their products.

The aim of the center is to bring the Korean industry at par with the global technology giants like Google and Amazon, who have gone ahead by 3-5 years by properly enrooting their big data.

Korea Big Data Centre is primarily created for small and medium size businesses, universities and citizens who wish to analyze their big data to find answers to their research or business issues.

Until the opening of KBiG, universities, research institutes and businesses in Korea were paralyzed when it came to securing and analyzing big data. They had to rely on limited IT infrastructure, they could create by themselves. Korea Big Data Centre will now provided the much needed shared service that everyone will benefit from.

According to an official from the Ministry of Science, the new centre is expected to be a “test bed” for big data analytics to foster research at universities. This, he says, is a basic solution for everyone to use the services in order to analyze big data.

Via: FutureGov

11/27/20131 commentRead More
New course to handle Big Data on Hadoop using R software

New course to handle Big Data on Hadoop using R software

Jigsaw Academy is introducing all new course in big data analytics using R and Hadoop. The course has been specifically designed to provide students’ knowledge and hone their skills to handle big data environment of Hadoop using the R software.

JigsawAcademyIt just been days we learnt of Cloudera and Udactiy partnership to offer open Hadoop and MapReduce courses. Course which have been specially designed to equip students with technical and analytical skills for a brighter career in emerging data market.Following the lead, Jigsaw Academy, a premier online analytics training academy, has introduced new courses in Big Data Analytics using R and Hadoop.

Jigsaw Academy has made a good name in online analytics training. It offers both intermediate and advanced level big data analytics courses. With a vision to extend its roots (as a premier academy), Jigsaw Academy has specifically designed their new course to provide everyone (in need) knowledge and help develop skills needed to deal with big data analytics on Hadoop using R software.

SaritaDigumarti, co-founder at Jigsaw Academy informs,

This new course is specifically designed for those looking to enhance their knowledge and skill sets in Big Data, specifically that of handling the big data environment of Hadoop using R software.

Who is the course for?

Since, Jigsaw Academy thrives on the continuous commitment to expand its offerings, the new course will really help global industry experts (who lack big data handling skills) garner significant expertise in the big data analytics environment. The primary target group for the course, being offered, are analytics experts who are wanting to learn and develop on their big data analytics skills.

It is also beneficial students planning to pursue a career in data science, or for those database professionals who plan to make an entry into the big data analytics industry.

Requirements for enrollment?

To attain an entry into the course, professionals and students are required to have working knowledge of R software. They should have a beginner’sunderstanding of statistics and SQL.

Those not versed with R will have to undergo a spate R skills course, which will be offered by Jigsaw Academy for free.

What to expect in and on completion of the course?

The course can be really beneficial for all the aforementioned type because the instructors at Jigsaw Academy will use real-time big data case studies. This will allows the instructors to showcase and clear the concepts of Hadoop in addition to providing training of application of big data technologies on large volume of data.

What to expect on completion?

  • A working knowledge of Hadoop
  • An ability to analyze big data using R software
  • Complete knowledge of big data analytics
  • And practical application of big data analytics

Via: PRWeb

To bridge market needs Udacity introduces paid big data courses

To bridge market needs Udacity introduces paid big data courses

Data analysts and data scientists are already in big demand. According to various surveys there will be an exponential requirement of big data experts around the globe in next three years, with 190,000 data science experts needed in US alone. Given this demand, data science courses are a good career option. In addition to various free online courses, Udacity has introduced paid courses to impart more refined and superior education. 

Choosing to launch full time experience in data science courses, Udacity recently announced two new initiatives for students. These include Data Science and Big Data Track and Paid Course Enrollment. The idea behind paid courses is to improve the focus on the free online courses that are otherwise very unprofessional.

Udacity paid big data courses

For the new courses, Udacity will make available both a new track and a new personal coach for students who register and enroll for the course by paying a monthly pre-determined fee. The courses comprise of lectures, project and auto-graded exercises in addition to the online content (which is available for free).

Significance of Udacity’s paid course

When asked about the importance of paid course, Udacity informed that enrolling in a paid course is like going to a great class, in contrast to a free online course which is like reading through a book aimlessly.

According to Udacity, the paid enrollment course will begin from January 2014. The charges under the scheme will be charged per month/per course. The good thing about the courses will be that these will continue without any rigid schedule. There will be no fixed start and end dates, so the students will have the flexibility to take as long as they desire to finish a particular course.

The real advantage that students will have with Udacity courses is that during the period of enrollment, students will have access to a personal coach who will assist them through the projects and guide them with advice and feedback. Students will also benefit with verified certificates, which will be presented for the accomplishment at the completion of the final project and exit interview.

Project work

Udacity’s paid enrollment scheme will lay extreme emphasis on project work. This is because a good job takes more than a good resume. Udacity course will thus require lot of project work, which will help students show off their projects as portfolio to their employers. This will increase their opportunity to get employment and enhance their practical knowledge.

Paid enrollment scheme – cost structure

Presently, Udacity has only released information about a few courses. Students who are enrolling in advance are being offered 30% discount. The discount in enrollment is available for four courses which are part of Data Science and Big Data Track. For entry level course the cost is $150 per month (discount available now) and for intermediate courses like Data Wrangling with MongoDB, which will be two month course, the cost is $200 per month (discount available now). In January a couple of more beginner level courses will be added.

Via: I-Programmer

Jut raises $20 million Series B funding to ride on top of big data arena

Jut raises $20 million Series B funding to ride on top of big data arena

Jut, a San Francisco-based Stealth mode platform developer for enterprise software to handle big data is slated to be a new entrant in the big data industry. To make its way, Jut has secured $20 million in a Series B funding.

Jut is developing big data software for enterprises, in a round led by Accel Partners with LightSpeed Venture Partners and Wing, Jut has raised funds to expand its engineering team and to have the first version of its product hit the market.Jut big data VC funding

Jut, with the funding, hopes to provide technology to enterprises by which they will be able to deal with the avalanche of data – called big data. In event, Jut articulates a vision to ride on top of the big data arena.

It is believed, and Apurva Dave, vice President of Marketing at Jut professes, by 2016 big data will cost over $200 million in IT spending. This bulk spending will not only be on the infrastructure to store and maintain big data, but it will be a spending on acquiring services and products that will help derive meaning from this mammoth data.

Jut will provide enterprises with big data infrastructure that will help them store, analyze and derive meaningful insight from big data.

The funded capital will be utilized by Jut to expand its engineering team. The capital will also be used in order to bring the first version of its product to the market. Jut is developing a development-based company with open-source culture, which thrives on the concept of data-based decision making to provide answers to the hardest question of big data.

Disney Has Been Really Creative with Big Data

Disney Has Been Really Creative with Big Data

Disney World is using big data to its advantage, a lesson small and big companies can take to use big data for their benefit. The “House of Mouse” is upgrading big data to the tune of $1 billion; the project aims to enhance user experience and make their visit to Disney World more awesome.

Watching people waiting in lines without a FastPass at the Walt Disney World can soon be a sight of the past. Disney is going to implement a new system that could make your visit to the House of Mouse highly personalized.


In order to take advantage of the magical realm of big data, Disney has introduced a new system – the MyMagic-project, wherein RFID equipped wristbands tell the Disney World employees what their guests in the park are up to. The MagicBand (bracelet) allows the guest wearing it to enter the Disney resort, buy food and other souvenirs, get on to the rides (in a predefined slot) and/or barge into the Disney hotel room – all by just touching the bracelet. Great idea, no?

Disney World - MyMagic Project

A Novel Beginning

Disney isn’t a very new player in the big data industry – Disney has been collecting a great deal of information for its marketing campaigns. This new step, however, is actually very novel. This is for the first time that Disney will track customer behavior at such minute level.

Complexity – When BIG DATA Enters the Picture 

Considering the scope that big data will open for Disney, it is important to understand the amount of data that will be generated by each person entering the world of Disney and how all this data will be analyzed and used to customize every person’s visit. Collection of data will begin right from the gate when someone will buy a ticket and places an order to use the MagicBand.

Now, imagine the convolution that Disney will face to optimize the data of each entrant to offer a tailor made experience for its customer. Addition problem will be the concern for privacy that people will show – Disney has out-rightly claimed that people will have the choice to control the amount of data that want to be shared.

MyMagic-project: Use and implementation

The all new system – MyMagic, is in the testing phase at the Disney World, according to reports; the MagicBand bracelet is being used to track all visitors attending the park annually.

How it works: The MagicBand bracelet works in two ways. Equipped with short range sensors, the bracelet allows visitors to make payments at the theme park or to open their hotel rooms. Embedded with long range sensors, the wristband allows Disney employees to keep track of what the guest is doing and where he/she is in the theme park.

Implementation benefits: There are two main purposes of the technology. One is to provide guests a customizable experience at the park and second is to increase the revenue by hoping to elongate guests’ visit time in order to get an opportunity to extract additional cash.

This innovative use of big data by Disney can be an example for many companies around the globe.