View Sidebar

Post Tagged with: Data

Identifying important information in Big Data to answer real world challenges

Identifying important information in Big Data to answer real world challenges

Over the past one year, knowingly or unknowingly, Big Data has become the biggest buzzword enterprises are finding hard to pass. Believe it or not, according to the current dependence on Big Data and its allied technologies, we can assume that Big Data is here to stay, and we all will have to use it to address our real world problems.

big-data

The way in which Big Data technologies have evolved in the real word enterprise goes on to show that even technologists and scientists who might have disparaged the word (Big Data) previously, will now be acknowledging it.

Like everything has loopholes, Big Data isn’t any different. Big Data problems are basically issues caused not because of the unavailability of data, but by the abundance of available data. There is so much influx of data that is rather impossible to know which piece of information is actually important and how different important information pieces can be put together for meaningful information.

Researching on a general way to understand complex systems and to answer the biggest question that Big Data can’t answer – ‘how to know what’s important in complex world?’ president and scientist of the New England Complex Systems Institute Yaneer Bar-Yam has devised a trick to identify patterns in largest scale of behavior. Bar-Yam has revealed his findings in the article titled “Beyond big data: Identifying important information for real world challenges”

According to Yaneer, to understand and address most social and biological challenges, it is important to frame a scientific inquiry with an idea to objectively conclude what is important or unimportant instead of amassing larger and larger sets of data.

Yaneer explains that the identified patterns of behavior determined from handful of information are the key to understanding a system and to inform how the behavior can be influenced in the future.

Yaneer Bar-Yam and his team have used the successful tested this approach by predicting various complex systems and real world challenges like market crashes, ethnic violence, food prices and many more biological and complex social systems.

Via: NECSI

Be Smart With Big Data

Be Smart With Big Data

smart dataSome companies get scared of big data. They think that since data is inherently dumb, a lot of it would be dumber still. But by being smart about big data, analysts can make sure that they get the most out of it. Handling big data can be a security risk and needs to be handled smartly.

The Present Way of Doing Things

Usually companies have one of three ways to handle data. They either go with the Heroic Model in which individuals take charge of requests and make decisions on their own without consulting with others. This model can work well for small businesses where individuals are usually aware of most situations across all areas of the business. But in bigger businesses, it can lead to confusion and chaos.

The Culture of Discipline on the other hand is one where individuals don’t make any decisions and follow a set of rules set by the management. Employees in this model can’t use data for their own decision making and just have to follow the processes set up for them.

The best way to handle data is to have a Data Smart Model in which data is managed on an evidence based management system. It is a combination of the first two methods and it works on a disciplined processing method but decision making is allowed at the individual level. This is the method that should be used to handle big data and it can result in smooth operation without much hassles.

How to Cultivate the Data Smart Culture

Certain steps need to be taken to create the data smart culture.

  • There should be a single source of truth. Decision making can be moved to the employee level but the guiding principles should be set from a single source.
  • Use ways to keep track of progress. Using a scorecard system, even on a daily basis, can help managers across different branches know how they are performing in relation to the other departments and they can then send in better data to record their progress.
  • Rules are important but there should be enough flexibility. Rules and guiding principles are needed but there should be flexibility to know when to bend the rules and when to break them. Sometimes what works in most parts of the country might not be best for a certain area. Businesses need to be able to adapt to such situations and change their rules accordingly.
  • Work on cultivating human resources. The people are the biggest asset of a company and it is important to educate them and provide them with the proper know-how to handle data. Managers need to be trained to educate the people working under them and give them a one to one engagement.

These steps can help businesses handle big data smartly and without much confusion. Every level needs to be trained to handle big data as the future is going to be all about big data.

IT Companies Should NOT Hesitate to Milk Big Data

IT Companies Should NOT Hesitate to Milk Big Data

Big Data is being utilized by businesses in every field to answer questions and predict the future with more reliability than the traditional methods that were used before big data analytics came along. It is surprising that IT isn’t utilizing big data as much as they should.

IT, like most other businesses, needs to predict the future in terms of surprise requirements, new opportunities and threats and worst case scenarios. A lot of these questions can be answered by using big data analysis. All parts of IT; operations, security, customer service, forecasting etc. can benefit by using big data.

IT has access to a lot of data in terms of logs, traces, emails, counters, feedback, polls etc. that it can use to solve critical problems related to predicting future scenarios.

Big data

The Usual Solution

The solution that IT has been using till now is to purchase packaged applications for their services and in case of a unique solution, they tend to integrate their own solutions to the applications with the help of their unique business know how.

The new idea is to use all the data collected, both internally and externally and apply big data analysis to it to move on to the next level.

It’s Already Started

A few IT companies have already started utilizing big data because they see the potential behind it. EMC IT is using big data to analyze their data and predict potential issues with their app delivery system.

Some companies are starting to use the huge amount of data they have collected for security. Already certain applications exist that analyze security data but by creating their own applications companies can come up with better solutions for their unique security requirement.

Companies can also use big data to forecast how much money they’d need to spend in the coming year on upgrading their capacity to store, compute and analyze data.

IT Always Leads the Way

It has been seen that the IT is always the one that takes on new technologies and systems and gains enough domain expertise in the area to help other branches of business later.

The data already exists and the tools for analysis of this data also exist. It’s just a matter of time before all IT companies start using big data in new and creative ways to solve unseen problems and predict the future. It is easy to get started and a lot of money is not required to get into big data. We are going to see a game changing utilization of data in the near future, just as soon as the IT sector wakes up and smells the data.

Apple’s Acquisition of Topsy Could be a Great Move

Apple’s Acquisition of Topsy Could be a Great Move

Apple has recently acquired Topsy Labs, a San Francisco based analytics firm, for $200 million. The Indian owned firm specializes in analyzing tweets and has an archive of over 400 billion tweets, starting from 2006 when Twitter was first launched. It also has a searchable database of all tweets ever sent, something that even Google doesn’t have, because of its partnership with Twitter.

topsyTopsy Labs was formed in 2007 by Vipul Ved Prakash, Rishab Aiyer Ghosh, Gary Iwatani and Justin Foutts. It collects and analyzes huge amount of data generated everyday on Twitter and allows its customers to make sense of all the noise. This buy could be a great move for Apple depending on how they use Topsy.

Apple hasn’t commented on what they intend to use Topsy for, but it is clear that with such a powerful tool, they have a lot of opportunities in front of them. Kristin Huguet, a spokesperson for Apple Inc, confirmed the acquisition but did not reveal Apple’s plans for Topsy.

“Apple buys smaller technology companies from time to time, and we generally do not discuss our purpose or plans,” Huguet said.

The Opportunities for Apple

The most obvious use is to get feedback about their products and services and market their products in a better way. They could also analyze key players on social networks and use them to spread the word about their services.

dataTopsy has developed many state of the art systems to deal with such huge amount of data obtained from social networks. They have filed patents for many systems and methods for customizing filtering and predictive crawling of social media. Apple could use this technology to analyze the data that they have been collecting through their devices and app store.

They could use it to improve the search feature of Siri, providing users with better search results by analyzing what recommendations their friends made on Twitter. Twitter is the most used social media platform on smart phones and getting access to all of its data along with the ability to analyse it, is definitely a great opportunity.

Competing with Google

Google has been trying to make their search engine more personalized based on data gathered from social media. Now Apple has a very strong tool to do the same. Both companies are spending big money on acquisitions in order to maintain their edge in the rapidly changing world of information technology. Apple spent $496 million in their last fiscal year on acquisitions while Google spent a whopping $1.4 billion.

Is Big Data a Threat to Your Privacy?

Is Big Data a Threat to Your Privacy?

Big Data is growing bigger every day and along with it the concern over invasion of privacy is also growing. Tracking all the data generated by your mobile and other devices and your interactions on social media, is beneficial for advertisers to tailor their ads to suit you. But there’s more to the story than that. Companies have now begun to come up with very creative ways to use real time data.

Let’s look at some interesting examples.

Smart Rubbish Bins in London

An advertising firm in London came up with the idea to use strategically placed dustbins to track the wifi signal of phones of the people passing by. They could use the serial number of the phones to track the movement of every individual. They could then use this data to show advertisements on the screen of these bins, that are targeted at the person passing by.

smartbins

Now even dustbins are becoming smart!

The officials have asked Renew, the responsible ad firm, to take down the smart dustbins as there has been a lot of concern about the invasion of privacy of the people.

Police Cars in Australia get Number Plate Recognition Cameras

The Aussies have come up with another great use of Big Data by using number plate recognition cameras that can read multiple number plates simultaneously and also search their database to find out all the information about that driver. They can tell if a car is stolen or if you have unpaid parking tickets just by looking at your car’s number plate.

police car

The hand of the law gets longer.

Are Such Examples a Threat to Your Privacy?

When CCTV cameras first came on the scene, the public responded to them with an outrage similar to what we see now in terms of Big Data. But once people got used to the new technology and saw the benefits in solving crimes and catching miscreants swiftly, the fears of Big Brother always watching them subsided.

The truth is that people will allow collection of any data as long as it is collected with their permission and it is used to create value for them. Instead of shoving ads in people’s faces, companies should try to find other ways to use Big Data, not only to reduce costs for the company but also to provide quality to the customer.

One great example to highlight the creative use of Big Data is the potential for insurance companies. Today all natural or man made calamities generate a lot of data in the social media.

data

Data about Hurricane Sandy

Insurance companies can use this data along with before and after images on Google Maps Street View, Flickr, Instagram etc. to find out how much destruction of property their clients have suffered.

torn houseThey can estimate the number and amount of claims that they will have to deal with. They can provide quick claim settlements to their customers which will be appreciated by all and people will readily agree to data collection if they are told of such rewards.

Great Opportunities

A Westpac survey showed that it only took 30 months for mobile usage to reach 1 Million as compared to 80 months it took for online usage to reach the 1 Million mark.

graphThis means that there are great opportunities available to use this rapidly growing Big Data but it will have to be done with care and while keeping the interests of the consumer in mind.

3 ways big data analytics is changing our lives

3 ways big data analytics is changing our lives

Big data has travelled a great deal from just being a buzzword only a year back. Service providers and analytics are fully engrossed in understanding the benefits which can be derived out of big data analytics. Since big data is everything we do every day to leave a digital trace, it can be analyzed to better our lives. Read on to know how big data analytics is changing our lives with each passing day.

big data changing livesWith the explosion of data in every form, from books to maps and from calls to apps, from advertisements to social network updates and from surveys to varying trends – we are leaving more and more digital traces with every digitalizing world. Since all this avalanche of data is touching our lives every day, it has become one of the mega trends that will impact (and is impacting) everyone in one way or the other.

Listed below are a few examples of how big data analytics is already changing our lives for good.

At home and office

There was a time when nothing was possible at home without human intervention, today we have small and seasoned manufactures developing devices that help us monitor every single thing at home, from elders to pets and from appliances to door locks, everything can be monitored from miles ways just by using a smartphone and an application.

Manufacturers have devised sensors to automate lighting and your appliances at home or in offices. You can turn on the lights or the air conditioner even before you walk into the office or home. You don’t have to stand up and turn of the light or walk to the washing machine to switch it on, a smartphone with dedicated app can do it for you from the comfort of your bed.

While driving or shopping

Auto manufacturers like Ford are using big data analytics to make their future vehicles more environmental friendly. In addition, there are companies and third-party developers who have developed applications to allow smartphones to send out location information, information about how fast you’re are driving, combined with the information of real time traffic to give you the best routes to avoid traffic, or give you information of the nearby gas station, restaurant, bank etc.

While shopping, the loyalty card of the store is combined with your purchase history and social media data to offer you discount coupons and personalized offers based on your loyalty to the store. This is really making shopping more fun, economical and very personalized.

Hospitals, healthcare and fitness

Doctors are maintaining record of patients to keep track of their medical history, for better, quicker and more pin point treatment. While, pediatric units in hospitals are live steaming heartbeats of premature and sick babies in the womb. Combining the information with historically data and based on analysis, doctors are now able to detect infections in babies even before they are born.

Fitness and healthcare have become the biggest market for electronic companies. Most manufacturers, including Nike have are coming out with fitness bands, smartwatches and pedometers etc to collect daily data of a person’s physical routine, calories, sleep patterns and heart rate etc., which is then sent wirelessly to smartphone, doctors and insurance companies to devise  better and more customized healthcare programs.

SiSense makes big data analytics possible on a Chip

SiSense makes big data analytics possible on a Chip

Thriving on the concept of ‘big data meets business intelligence,’ a company in Israel has worked out a way to allow enterprises to put analytics in cache memory of a CPU in order minimize cost of hardware purchase that enterprises incur. SiSense big data analytics on a chip If you were to ask someone, what is big data’s biggest drawback? They would outrightly claim it as ‘more hardware’meaning more expenditure! Since, organizations see their big data levels rise more than twice each year, they need ways to store this data. Not just store, but also process, search and analyze data from time to time. All this requires IT infrastructure, which involves a lot of cost. As an innovative stride forward to help organizations save considerably on this cost of hardware, SiSense, Big Data Analytics Companyhas designed a way to do analytics in the cache memory of Central Processing Unit – Analytics on a Chip i.e. The idea of rethinking business intelligence – with high speed and smaller sets on multicore processors began way back in April when SiSense received $10 million in Series B funding from Battery Ventures along with Opus Capital and Genesis Partners. SiSense has basically worked out a technique to make analytic software work on multicore processors, say a parallel computer cluster on a Chip. Explaining the efficiency of the system, SiSense CTO EldadFarkash says that their big data analytics on a chip system can run queries against almost 20 terabytes of data on the Chip. According to Farkash, their technology works on Intel and AMD multicore 64-bit architectures and is probably the first to support Intel’s new Haswell architecture. Explaining the applicable use of the technology in the future, Farkash says an analytics system like theirs would be something we will carry in the palm of our hands, it will be a system that will work on new age iPads and Android tablets with terabytes of storage. Seen from a broader prospective, SiSense could be a deviation from high-performance computing ofbig data on Hadoop Hive. This could really save enterprises a lot that they end up spending on maintaining their data and analyzing it using various platforms. SiSense makes for a good way to diverge big data analytics beyond Hadoop hives, your take? Via: InformationWeek

10 Must-Know Facts about Big Data

10 Must-Know Facts about Big Data

With the increasing awareness in big data levels, enterprises around the world are realizing the potential of this avalanche of data analytics and are finding ways to unlock business value from all this data.

Big data has become a buzzword in the world of technology of late. Analytics and service providers are busy understanding the various benefits that can be derived out of big data by collecting, analyzing and utilizing this data. But the questions remain – where does this data come from? How can an enterprise benefit from it?  To answer your inquisitiveness we have listed below must-know facts about big data.

bigdata facts

  1. Data experts believe that big data is still in a very nascent stage, and most of the enterprises are thus still contemplating on whether they should adapt, or continue their wait and watch strategy. Everyone does understand that the trend will change and big data has the potential to change the way we interpret data today.
  2. Since big data has become an integral part of a solution to world problems, it is generally improving power consumption, transportation and making social networks ever more efficient to use.
  3. It is estimated that the current global storage capacity for digital information has reached about 1,200 exabytes. To estimate how much that is – if all this digital information was to be placed on CD-ROMs stacked up in piles, it would form approximately five separate piles, each reaching to the moon.
  4. Another exciting fact about big data is that there are nearly as many bits of digital information as much there are stars in the universe. This is the kind of big data we are talking about.
  5. With each passing hour, people around the world consume enough digital information to fill 7 million DVDs. To explain how much that would be – if you put all 7 million DVDs side by side, they would scale up to the Mount Everest about 95 times.
  6. We cannot think of a place where we wouldn’t see people using mobile phones. Using mobile phones is so normal that it is hard to believe that there are still places in the world devoid of mobile phones. Yet, the reach of mobile phones has, and is spreading globally like wild fire. The reach is so magnanimous that according to a report, the number of active cellphones will reach an estimate of 7.3 billion by 2014. This number may be more than the number of people on earth, but it doesn’t mean that everyone on the planet has a phone; this only represents number of users with multiple phones. Internet users on mobile phones and other mobile devices together make up for 36 percent of the world population, which creates billions of digital data pieces every day.
  7. An estimated 247 billion e-mails are generated each day, out of which 80 percent are recorded as spam. In addition to e-mails, Facebook is the second largest data cruncher on the internet. An estimated 30 billion pieces of data are shared on Facebook on a daily basis.
  8. Considering the size of data that is being generated, there are currently over 500,000 data centers across the globe. These are large enough to fill about 5,950 football fields.
  9. In the digital world, about 75 percent of digital information is produced individuals.
  10. Given the rate at which digital data is multiplying, IT departments across the globe will require ten times more servers by the year 2020. In addition, considering the data growth almost twice the number of current data analysts will be required.
11/11/20131 commentRead More