View Sidebar

Archive for category: Blog

Oracle Launches 5th Gen Database Machine

Oracle Launches 5th Gen Database Machine

Oracle Exadata Database Machine X4, the 5th gen database machine form Oracle is a revolutionary step in the field of database management. Keep on reading to know what it has to offer!

Oracle Launches 5th Gen Database MachineOracle recently launched Exadata Database Machine X4. It has hi-tech hardware and software that can increase capacity, boost performance and maintain quality and efficiency of service for database operation.

The update focuses mainly on the optimization of Online Transaction Processing (OLTP). The machine is mainly aimed at providing businesses with a permanent solution for all major database challenges and has advantages like Data Warehousing and Database as a Service (DBaaS).

Oracle Exadata Database Machine X4 – Features

  • The machine is the 5th gen of Oracle Exadata that was launched in the year 2008.
  • It is a fifth gen machine featuring improvements that focus on improved performance as well as quality of service for OLTP, Database as a Service and Data Warehousing.
  • It uses high speed flash compression and larger physical flash in perfect combination to increase the capacity of flash memory that eventually accelerates the performance of OLTP-based work.
  • The latest Flash Caching algorithms help accelerate the performance of all workloads in Data Warehousing.
  • Many databases can be merged with help of the Database as a Service design because of extreme capacity and performance. This will help businesses improve on quickness and more importantly, reduce costs.
Hadoop Security: Present and Future

Hadoop Security: Present and Future

Where the current level of Hadoop system can be relied upon for data protection and processing, there is still a need to improve Hadoop security to ensure foolproof big data security for coming times. To stay updated about the scope of a secure Hadoop cluster today and in times to come, one needs to know a few important things about it.

Hadoop Security Present and FutureSecurity is the foremost agenda that represents almost all major requirements within an organization, especially when it is about tasks like big-data processing. Hadoop registered a remarkable progress in last couple of years and has successfully addressed the most common worries like authorization, authenticity and above all, data protection. With more security enhanced Handoop clusters in the pipeline, though using the systems are banking upon the safety of all vital data in the future also.

Hadoop currently is engaged at the cutting edge to provide secure support to countless financial service applications and big private healthcare projects that operate in a high security-sensitive environment. Recent upgrades of Hadoop systems meet the key requirements of organizations demanding some of the world’s toughest security norms. With all the tight security controls incorporated in Handoop, the final objective remains flexibility and smooth data processing for now and in the future.

Hadoop Security Controls Dec 2013

 Security Controls for Hadoop at Present

Securing a Handoop cluster presents certain both small and big, which includes its distributed nature that to a large extent is even responsible for its success. For securing a system, a layered approach is the best and distribution happens to be one of the most complex barriers to it.

Following are the major layers that are in place to secure a cluster:

Authentication

It is responsible for verifying the identity of both a system and a user accessing it. Pseudo authentication and Kerberos are the two authentication modes Hadoop is providing. While the first takes care of the trust among users, the latter secures the overall Hadoop cluster.

Authorization

Authorization represents access freedom for users and a system. Hadoop relies on resource-level access control, file permissions in HDFS and offers authorization and a service-level access control.

Accounting

Accounting makes it possible to track resource use in a system. MapReduce and HDFS that are the parts of Apache Hadoop offer base audit support. Apache Oozie functions as a workflow engine and offers audit trail for all services.

Data Protection 

This takes care of privacy of information. HDP protects the data in motion and HDFS holds up encryption at operating-system levels.

Security Controls for Hadoop in Future

Newer innovations in Hadoop security are focusing mainly on making various security frameworks to work in collaboration so that they can be easily managed. Here’s what Hadoop security system is going to be big at:

Granular Authorization and Enhanced Authentication

Verification technique in most Hadoop modules is in the process of being improved. This is mainly developed and fortified mainly because most users are demanding security hardened authorization model. Token-based validation will soon replace Kerberos to enhance the authentication process.

Encryption Data Protection and Improved Accounting

A more advanced encryption algorithm is a must for most channels. The focus would be on better encryption, mostly through HBase, HDFS and Hive. Another important step is going to be high-tech audit record correlation for easier reporting. With this system, the auditor would be able to predict the sequence of Hadoop component operations without having to take help from any external tools.

Be Smart With Big Data

Be Smart With Big Data

smart dataSome companies get scared of big data. They think that since data is inherently dumb, a lot of it would be dumber still. But by being smart about big data, analysts can make sure that they get the most out of it. Handling big data can be a security risk and needs to be handled smartly.

The Present Way of Doing Things

Usually companies have one of three ways to handle data. They either go with the Heroic Model in which individuals take charge of requests and make decisions on their own without consulting with others. This model can work well for small businesses where individuals are usually aware of most situations across all areas of the business. But in bigger businesses, it can lead to confusion and chaos.

The Culture of Discipline on the other hand is one where individuals don’t make any decisions and follow a set of rules set by the management. Employees in this model can’t use data for their own decision making and just have to follow the processes set up for them.

The best way to handle data is to have a Data Smart Model in which data is managed on an evidence based management system. It is a combination of the first two methods and it works on a disciplined processing method but decision making is allowed at the individual level. This is the method that should be used to handle big data and it can result in smooth operation without much hassles.

How to Cultivate the Data Smart Culture

Certain steps need to be taken to create the data smart culture.

  • There should be a single source of truth. Decision making can be moved to the employee level but the guiding principles should be set from a single source.
  • Use ways to keep track of progress. Using a scorecard system, even on a daily basis, can help managers across different branches know how they are performing in relation to the other departments and they can then send in better data to record their progress.
  • Rules are important but there should be enough flexibility. Rules and guiding principles are needed but there should be flexibility to know when to bend the rules and when to break them. Sometimes what works in most parts of the country might not be best for a certain area. Businesses need to be able to adapt to such situations and change their rules accordingly.
  • Work on cultivating human resources. The people are the biggest asset of a company and it is important to educate them and provide them with the proper know-how to handle data. Managers need to be trained to educate the people working under them and give them a one to one engagement.

These steps can help businesses handle big data smartly and without much confusion. Every level needs to be trained to handle big data as the future is going to be all about big data.

9 Recent Surveys About Big Data

9 Recent Surveys About Big Data

big data surveyBig Data is the big word right now and many surveys have been conducted to find out just how big is Big Data. Take a look at the highlights of 9 such surveys to find out where big data is headed.

1. CompTIA

CompTIA, the IT association, surveyed 500 businesses and found out that:

  • 42% of businesses admitted that they have some type of big data initiative going on.
  • 93% said that data was critical for their business.
  • 18% thought that their business was ready for big data.

2. EMA and 9Sight Consulting

259 businesses and professionals were surveyed in this end user research. They found that:

  • 68% of the companies have at least 2 projects in their big data initiative.
  • 34% companies are using big data implementations in production.
  • 39% companies identified speeding operational time for analytics to be the number one driver for big data initiatives. Other drivers included competitive advantage with data use in business solutions (34%) and business requirements for higher levels of advanced analytics (31%).

3. Tech Pro Research

Tech Pro Research surveyed 144 businesses about the financial side of big data.

  • 54% have no interest in implementing big data initiatives. 8% have implemented some form of big data initiative, 12% are implementing and 26% are planning on implementation.
  • 82% of those who have implemented big data initiatives report seeing some form of payoff while only 4% believe they haven’t seen any benefits.

4. Gartner

They surveyed 720 members of the Gartner Research Circle.

  • 64% of companies are investing or planning to invest in big data.
  • But less than 8% have already deployed some form of initiative.

5. TEKsystems

They surveyed more than 2000 IT professionals and 1500 IT leaders. They found in their report:

  • 90% of IT leaders and 84% of IT professionals believe in big data as a good investment for time and money.
  • 14% of IT leaders said that big data is regularly applied in their businesses.
  • 66% of IT leaders and 53% of IT professionals said that their data is stored in disparate systems.
  • 60% of IT leaders said that their is no accountability for data quality.
  • At least 50% of IT leaders were not sure about the validity of their data.
  • 81% of IT leaders accepted that they do not have the adequate manpower with the right skill sets to implement big data initiatives.

6. Bain

Bain studied 400 large companies and found good results about big data.

  • They found that the companies that have big data analytics capabilities were outperforming their competition.
  • They were twice as likely to have better quarterly financial performance.
  • Five times more likely to make faster decisions.
  • Three times as likely to execute decisions.
  • And twice as likely to use data in their decision making process.

7. BCG

BCG surveyed 10,000 consumers in 20 countries worldwide. They found that:

  • 75% of consumers are concerned about privacy of their data.
  • The young generation is just as concerned about privacy as older generations.
  • Consumers will allow use of data as long as they trust the business with their data.

8. IBM

IBM studied 900 businesses from around the world. They found that the companies that were outperforming their peers were:

  • 166% more likely to make decisions based on data.
  • 2.2 times more likely to have a clear path for big data analytics in their organization.
  • driven by growth as the main source of value from data analytics.
  • measuring the impact of investment in analytics.

9. Forbes Market Insights

Rocket Fuel sponsored this study by Forbes Market Insights of 211 senior marketers.

  • The marketers that used big data at least half of the time in their campaigns said they exceeded their goals 3 out of 5 times.
  • Those who used data less than half of the time achieved similar results only 1 out of 3 times.
  • 92% of companies who used big data, exceeded their goals and only 5% fell short.

From such surveys it is clear that it is still early days for big data. Those who have taken the initiative have found dividends in big data while some still remain skeptical. Information taken from the article on Forbes.

Hadoop Can Come Handy Even When You are Not Dealing with Big Data

Hadoop Can Come Handy Even When You are Not Dealing with Big Data

Hadoop was developed to cater to the needs of web and media companies for managing big data. But even if you don’t have to deal with big data, you can still use Hadoop in many ways to enhance your data and resource management. Today Hadoop is being used by almost every business, whether they have big data or small, to manage their data.

The Main Features of Hadoop

The main feature of Hadoop is the HDFS storage system. HDFS stands for Hadoop Distributed File System that operates on low cost hardware.

MapReduce was developed for resource management and data processing but with Hadoop 2.0 it has been left just to focus on data processing while YARN is used for resource management.

These features of Hadoop can be utilized in many innovative ways by big and small businesses.

Data Archive

One straightforward use of Hadoop is to archive data files. Since HDFS runs on commodity hardware it is simple and cheap to scale so businesses can start small and expand as their business grows. They can store all their data at a very low cost.

Instead of destroying data after the regulatory period is over, companies can store decades of data and analyze it in real time to help their decision making process.

Data Staging Area

Traditionally ETL tools are used for extracting and transforming data. When Hadoop came to the scene, it could have killed ETL forever if ETL providers hadn’t been smart enough to provide HDFS connectors so that Hadoop could be used along with their ETL software.

By using Hadoop you can store the application data and the transformed data in the same place. This makes it easier to process the data at a later time and reduces the time to process the data. Hadoop can help ETL in improving data processing.

Data Processing

Instead of sending data to the warehouse and then use costly resources to update it in the warehouse, you can use Hadoop and its MapReduce function to process and update it before it goes to the warehouse. Hadoop’s low cost processing power can be used not just for your warehouse data but for other operational and analytical systems as well.

HadoopHadoop is a very powerful tool that can help all businesses to handle their data in a better way. You don’t have to be sitting on top of big data to use Hadoop. You can start even when you have small data and Hadoop will let you collect decades of data till it becomes big data and then you can start making use of all this data by using big data analytics.

IT Companies Should NOT Hesitate to Milk Big Data

IT Companies Should NOT Hesitate to Milk Big Data

Big Data is being utilized by businesses in every field to answer questions and predict the future with more reliability than the traditional methods that were used before big data analytics came along. It is surprising that IT isn’t utilizing big data as much as they should.

IT, like most other businesses, needs to predict the future in terms of surprise requirements, new opportunities and threats and worst case scenarios. A lot of these questions can be answered by using big data analysis. All parts of IT; operations, security, customer service, forecasting etc. can benefit by using big data.

IT has access to a lot of data in terms of logs, traces, emails, counters, feedback, polls etc. that it can use to solve critical problems related to predicting future scenarios.

Big data

The Usual Solution

The solution that IT has been using till now is to purchase packaged applications for their services and in case of a unique solution, they tend to integrate their own solutions to the applications with the help of their unique business know how.

The new idea is to use all the data collected, both internally and externally and apply big data analysis to it to move on to the next level.

It’s Already Started

A few IT companies have already started utilizing big data because they see the potential behind it. EMC IT is using big data to analyze their data and predict potential issues with their app delivery system.

Some companies are starting to use the huge amount of data they have collected for security. Already certain applications exist that analyze security data but by creating their own applications companies can come up with better solutions for their unique security requirement.

Companies can also use big data to forecast how much money they’d need to spend in the coming year on upgrading their capacity to store, compute and analyze data.

IT Always Leads the Way

It has been seen that the IT is always the one that takes on new technologies and systems and gains enough domain expertise in the area to help other branches of business later.

The data already exists and the tools for analysis of this data also exist. It’s just a matter of time before all IT companies start using big data in new and creative ways to solve unseen problems and predict the future. It is easy to get started and a lot of money is not required to get into big data. We are going to see a game changing utilization of data in the near future, just as soon as the IT sector wakes up and smells the data.

Apple’s Acquisition of Topsy Could be a Great Move

Apple’s Acquisition of Topsy Could be a Great Move

Apple has recently acquired Topsy Labs, a San Francisco based analytics firm, for $200 million. The Indian owned firm specializes in analyzing tweets and has an archive of over 400 billion tweets, starting from 2006 when Twitter was first launched. It also has a searchable database of all tweets ever sent, something that even Google doesn’t have, because of its partnership with Twitter.

topsyTopsy Labs was formed in 2007 by Vipul Ved Prakash, Rishab Aiyer Ghosh, Gary Iwatani and Justin Foutts. It collects and analyzes huge amount of data generated everyday on Twitter and allows its customers to make sense of all the noise. This buy could be a great move for Apple depending on how they use Topsy.

Apple hasn’t commented on what they intend to use Topsy for, but it is clear that with such a powerful tool, they have a lot of opportunities in front of them. Kristin Huguet, a spokesperson for Apple Inc, confirmed the acquisition but did not reveal Apple’s plans for Topsy.

“Apple buys smaller technology companies from time to time, and we generally do not discuss our purpose or plans,” Huguet said.

The Opportunities for Apple

The most obvious use is to get feedback about their products and services and market their products in a better way. They could also analyze key players on social networks and use them to spread the word about their services.

dataTopsy has developed many state of the art systems to deal with such huge amount of data obtained from social networks. They have filed patents for many systems and methods for customizing filtering and predictive crawling of social media. Apple could use this technology to analyze the data that they have been collecting through their devices and app store.

They could use it to improve the search feature of Siri, providing users with better search results by analyzing what recommendations their friends made on Twitter. Twitter is the most used social media platform on smart phones and getting access to all of its data along with the ability to analyse it, is definitely a great opportunity.

Competing with Google

Google has been trying to make their search engine more personalized based on data gathered from social media. Now Apple has a very strong tool to do the same. Both companies are spending big money on acquisitions in order to maintain their edge in the rapidly changing world of information technology. Apple spent $496 million in their last fiscal year on acquisitions while Google spent a whopping $1.4 billion.

Is Big Data a Threat to Your Privacy?

Is Big Data a Threat to Your Privacy?

Big Data is growing bigger every day and along with it the concern over invasion of privacy is also growing. Tracking all the data generated by your mobile and other devices and your interactions on social media, is beneficial for advertisers to tailor their ads to suit you. But there’s more to the story than that. Companies have now begun to come up with very creative ways to use real time data.

Let’s look at some interesting examples.

Smart Rubbish Bins in London

An advertising firm in London came up with the idea to use strategically placed dustbins to track the wifi signal of phones of the people passing by. They could use the serial number of the phones to track the movement of every individual. They could then use this data to show advertisements on the screen of these bins, that are targeted at the person passing by.

smartbins

Now even dustbins are becoming smart!

The officials have asked Renew, the responsible ad firm, to take down the smart dustbins as there has been a lot of concern about the invasion of privacy of the people.

Police Cars in Australia get Number Plate Recognition Cameras

The Aussies have come up with another great use of Big Data by using number plate recognition cameras that can read multiple number plates simultaneously and also search their database to find out all the information about that driver. They can tell if a car is stolen or if you have unpaid parking tickets just by looking at your car’s number plate.

police car

The hand of the law gets longer.

Are Such Examples a Threat to Your Privacy?

When CCTV cameras first came on the scene, the public responded to them with an outrage similar to what we see now in terms of Big Data. But once people got used to the new technology and saw the benefits in solving crimes and catching miscreants swiftly, the fears of Big Brother always watching them subsided.

The truth is that people will allow collection of any data as long as it is collected with their permission and it is used to create value for them. Instead of shoving ads in people’s faces, companies should try to find other ways to use Big Data, not only to reduce costs for the company but also to provide quality to the customer.

One great example to highlight the creative use of Big Data is the potential for insurance companies. Today all natural or man made calamities generate a lot of data in the social media.

data

Data about Hurricane Sandy

Insurance companies can use this data along with before and after images on Google Maps Street View, Flickr, Instagram etc. to find out how much destruction of property their clients have suffered.

torn houseThey can estimate the number and amount of claims that they will have to deal with. They can provide quick claim settlements to their customers which will be appreciated by all and people will readily agree to data collection if they are told of such rewards.

Great Opportunities

A Westpac survey showed that it only took 30 months for mobile usage to reach 1 Million as compared to 80 months it took for online usage to reach the 1 Million mark.

graphThis means that there are great opportunities available to use this rapidly growing Big Data but it will have to be done with care and while keeping the interests of the consumer in mind.

Apps may minimize need for big data scientists

Apps may minimize need for big data scientists

If technology can be backed, then soon we could have a new breed of applications that would allow businesses to create their own big data apps in cloud, thus minimizing the need for IT and big data scientists. 

big data appsBig data, a word that was brushed away as a buzz word by many technology scientists has made it presence felt in a big way. Even the scientist that dared to believe it as only a buzz world have acknowledged the importance of big data for enterprises.

Considering the hype around big data, recently a survey was conducting by SAS institute. The survey concluded that there is a significant rise in demand for qualified data scientists, who can find decisive results from the avalanche of data.

The report revealed that there is an expected 243% increase in demand for big data specialists in UK alone in the next five years.

The opportunity for big data scientists is enormous, since there isn’t enough number of experts on the ground. But is this demand only hype on the big data expansion, or is it time for universities and institutes to churn out armies of qualified big data experts?

According to an article by Successful Workplace co-founder and marketing executive, Chris Taylor, on Wired, there are good enough scientists around already, and as more institutes and courses spring up, producing data analysts will be an automated effect.

An infographic – world needs more data scientists, points out that ‘data scientist job postings increased 15,000 percent between 2011 and 2012 alone’. And people seeking career in big data technologies could soon become the most sought-after people in the IT industry.

However, according to Chris Tylor, the word big data scientist has just been coined a few years back and thus there is so much demand for it. He further that numbers don’t really add up, he is off the view that data scientists have always been there in the IT industry, it just the role is more demanding this time.

According to Taylor and a point that really strikes us is that, given the exponential growth of technology, data scientists could soon find themselves replaced with new applications devised to simplify the process of big data analytics.

New apps could be launched soon, which will allow businesses, governments and individuals to create their own big data apps in the cloud with much lower number of data scientists. Citing this Taylor believes enterprises should refrain from rushing into hiring too many data scientists for now.

Via: SiliconAngle

Cloud computing and big data enable real-time decisions

Cloud computing and big data enable real-time decisions

Big data analytics and cloud computing together have a potential to act as a huge springboard to real-time decision making. As big data and cloud become increasingly intervened, it is leading to more efficient and accurate results in real-time. 

A Close relationship between big data and cloud computing coupled with real-time processing capabilities will give birth to data analytics proficient in producing real-time results that can change the way companies build and market products. This sentiment was recently expressed by Amazon Chief Technology Officer, Werner Vogels while talking to The Guardian.

Cloud and BigData

Tools and technologies of big data offer new automated ways to condense large amounts of data into understandable format instantly. Unlike the conventional business intelligence, which had limitation of being futuristic, big data has the ability to not just analyze data about what has happened, but also has the potential to process currently produced data in real-time.

In Vogels’ view, big data and cloud computing share a very close relationship, since it requires no limits to store and compute. And when this ability is combined with real-time processing, data analytics will rise instantly to produce real-time results for companies using it. This will allow the enterprises to take real-time decisions based on real-time processing and analysis of data – business will be able to make decisions based on current status and not on the past information that was the case until.

Imagine a scenario where companies could take manufacturing and marketing decisions based on the most current data, which is stored in cloud and analyzed by big data tools. How effective will be the decisions that’ll be taken on the information available right now?

This is the kind of radical impact business decision making could experience if they can figure out ways to use cloud computing and big data in harmony.

It is evident that cloud and big data make a good team, considering the fact that you can store as much data as possible on cloud, as and when it comes. But there is a point where businesses will have to consider when this collaboration is becoming too expensive for them.