Wall Street loves a good reinvention story. The tough part is finding a happy ending. All the plots seem to go something like this: Every company wants to convince us it’s becoming a tech company–even if it kills them. To be fair, an increasing number of companies are at least dabbling in new tech ventures to improve operations, Howard Dresner, president of Dresner Advisory Services, told me. The boom in vendors offering affordable ways to crunch data or utilize cloud computing, for instance, unlocks new strategies for companies across a wide variety of industries.
Cloud business intelligence and analytics systems were greatly outnumbered by on-premises BI environments until recently. But they've become much more prevalent, according to Gartner. Most new deployments are cloud-based, the consulting firm said in its 2018 Magic Quadrant report on analytics and BI platforms. The report, published in February 2018, noted that some BI vendors now only offer their software in the cloud, while others have adopted a cloud-first approach to adding new features. Likewise, market researcher Dresner Advisory Services noted in a March 2018 report that its annual surveys of enterprise users showed "a steady progression of cloud BI awareness and adoption," to the point where reported use of the cloud for BI and analytics was just under 50% in the 2018 survey -- nearly double the usage rate in 2016.
IT organizations are starting to signal their intent to consolidate all the analytics applications they need to support around a common pool of “big data” whenever possible. A recent report from Dresner Advisory Services finds 60 percent of respondents would prefer to have a single analytical data infrastructure (ADI) platform, with the majority of those respondents preferring that platform to reside in the cloud. The challenge most organizations have historically faced is that it’s been relatively simple for most lines of business to set up their own analytics applications, as the cost of deploying these applications locally or accessing them as a cloud service has continued to decline. As more organizations start to view data as a strategic asset, there’s a push to centralize it. The primary driver of that shift will be the rise of artificial intelligence (AI) applications that require access to massive amounts of data to accurately create an AI model that automates a business process. While that shift is still relatively nascent, the critical mass of data being aggregated will eventually create enough gravity to pull all the analytics applications of an organization within the scope of the big data repository being employed to drive AI applications.
In a recent conversation on the Early Adopter Podcast, I spoke with Howard Dresner and Chris von Simson from Dresner Advisory Services. They recently completed a report about IT analytics, and their research sheds light on the role of productized analytics for IT. Our conversation (see transcript here) offered yet another perspective on how productized analytics can be a catalyst for better use of data in many companies. I’ve seen all of these levels represented in different kinds of analytics products. But in speaking with Dresner and von Simson, I was interested in how these products can lead to increased awareness from analytics that results in better outcomes in IT. In adopting productized analytics solutions, IT managers are also essentially adopting best practices for managing the complex technology landscape at most companies.
IoT is complex with immense amounts of data that can overwhelm enterprises. Leading organizations are applying analytics that will help make sense of it all. The Internet of Things (IoT) is evolving into more than just a gigantic global glob of connected sensors — it is now rapidly becoming intelligent as well. That’s where it is beginning to truly deliver economic benefits to businesses with the wherewithal to tap into this global brain. That’s the gist of a recent study published by Dresner Advisory Services, which finds intelligent IoT is beginning to make an impact in selected areas of organizations. The study’s authors find that 32% of companies are making investments in IoT today, with another 48% planning investments over the course of the next two years. A majority of these investments will be focused upon IoT data analysis, versus IoT infrastructure or an IoT data supply chain.
Big data adoption in enterprises soared from 17% in 2015 to 59% in 2018, reaching a Compound Annual Growth Rate (CAGR) of 36%. Amazon Web Services (AWS) S3, Spark SQL, Hive, and HDFS are the most popular big data access methods with AWS S3 is growing most quickly in 2018. Apache Spark MLib and Tensorflow are the most-adopted big data analytics and machine learning technologies in enterprises today. Cloudera, Amazon EMR, Hortonworks, and MAP/R are the most popular big data distributions by enterprises in 2018. These and many other fascinating insights are from Dresner Advisory Services’ 2018 Big Data Analytics Market Study part of their Wisdom of Crowds® series of research. This is the 4th annual Big Data Analytics report the firm has done, examining end-user trends and intentions surrounding big data analytics, defined as systems that enable end-user access to and analysis of data contained and managed within the Hadoop ecosystem.
Business Intelligence (BI) embedded as a feature in applications is growing in importance, according to the 2018 Edition Embedded Business Intelligence Market Study by Dresner Advisory Services, LLC, Nashua, N.H. Dresner has been studying embedded BI for six years and 2018 was the first year that embedded BI was rated as "very important" among those surveyed. "Combined 'critical' and 'very important' scores also reach a new high in 2018," the survey noted. BI tools and technologies include query and reporting, OLAP (online analytical processing), data mining and advanced analytics, end-user tools for ad hoc query and analysis, and dashboards for performance monitoring, according to Howard Dresner, chief research officer for the firm.
Go ahead, get that second beer. Call your Mom. You have time. This holiday season, location intelligence and indoor mapping technology is a traveler’s best friend. Now, simply by opening up your airline app, you’ll be able to see in digital reality, exactly how many minutes your security line wait is, how long the wait is at Starbucks and where the nearest airport lounge is. While airports are an obvious use case for smart building technology now and one we are beginning to learn a lot from, in the near future, every CIO will be tasked with designing smart buildings. According to Dresner Advisory Services’ 2018 Location Intelligence Market Study, 66 percent of enterprises rank location intelligence as either critical or very important to ongoing revenue growth strategies, and it’s no wonder why.
SHARE THIS: Click to share on Twitter (Opens in new window)Click to share on LinkedIn (Opens in new window)Click to share on Facebook (Opens in new window)Click to share on Reddit (Opens in new window) By now, most organizations are at the very least familiar with the Internet of Things (IoT) as a concept. However, adoption of IoT has been uneven at best. A new report published by Dresner Advisory Services finds only 32 percent of respondents are making investments in IoT today. The good news is the survey finds another 48 percent plan to make investments over the course of the next two years. At a recent Ingram Micro ONE 2018 conference, Eric Hembree, director of IoT U.S. for Ingram Micro, noted that only about two percent of IoT projects have made it beyond the proof of concept (PoC) stage. It’s clearly going to be a while before there is a critical mass of organizations that have moved IoT projects into production. But managed service providers (MSPs) should be planning today for spectrum of services opportunities that will emerge as more devices are connected to the Internet over the next three plus years.
Sales, Marketing and Operations are most active early adopters of IoT today; Early adopters most often initiate pilots to drive revenue and gain operational efficiencies faster than anticipated; 32% of enterprises are investing in IoT, and 48% are planning to in 2019; IoT early adopters lead their industries in advanced and predictive analytics adoption. These and many other fascinating insights are from Dresner Advisory Services’ latest report, 2018 IoT Intelligence® Market Study, in its 4th year of publication. The study concentrates on end-user interest in and demand for business intelligence in IoT. The study also examines key related technologies such as location intelligence, end-user data preparation, cloud computing, advanced and predictive analytics, and big data analytics. “While the market is still in an early stage, we believe that IoT Intelligence, the means to understand and leverage IoT data, will continue to expand as organizations mature in their collection and leverage of sensor level data,” said Howard Dresner, founder, and chief research officer at Dresner Advisory Services. 70% of respondents work at North American organizations (including the United States, Canada, and Puerto Rico). EMEA accounts for about 20%, and the remainder is distributed across Asia-Pacific and Latin America. Please see pages 11, 15 through 18 of the study for specifics regarding the methodology and respondent demographics.
Increasingly, businesses are turning to geographic information systems (GIS) to develop new markets, solve supply chain problems, predict growth, improve efficiency of routes and services, gain competitive edge, and suggest future strategies. And lately, colleges have been taking note. From Pennsylvania State University to the University of Redlands in California, the academic world is beginning to recognize the need to develop GIS skills through undergraduate and MBA-level courses. Though not always highlighted or widely understood in the corporate world, location intelligence–often derived from GIS–plays a crucial role in the success of tens of thousands of businesses and organizations around the globe. Those with GIS skills within their data and business analysis capabilities are finding themselves at the center of vital discussions about the direction and operations of their companies. In fact, more than 65 percent of businesses consider location intelligence critical or very important to revenue growth, according to a survey by Dresner Advisory Services.
The amount of data we create is staggering. On average, people send nearly 300 billion emails and perform more than one billion Google searches on any single day. And these are just two of the many ways in which humans interact with technology. As consumers produce more information, the opportunity for companies to make data-driven decisions becomes greater—but only if they can make sense of it all. Thirty-two percent of executives agree that massive amounts of data have made things worse. For companies, this has made Business Intelligence (BI) more relevant than ever. While BI isn’t new, companies are now focused on the democratization of data, or empowering individual users across the organization to get the right insights when they need them most. And self-service is top-of-mind. According to the Dresner Advisory Service’s 2018 Wisdom of Crowds® Business Intelligence Market Study, dashboards, reporting and end-user self-service were named as the top three most important initiatives strategic to BI this year.
A new report published this week by Dresner Advisory Services sheds some light on how IT organizations want to apply advanced analytics to optimize IT operations. The 2018 IT Analytics Market Study finds that the top four management metrics in terms of importance are software license and utilization analysis, vendor license compliance reporting, alignment of headcount, cloud license optimization, and optimizing time spent on tasks versus project tasks. Within the context of operations, however, the report finds that service experience and service level agreement (SLA) performance along with incident root cause analysis are the top two requirements for IT organizations. Today varied approaches are being used to analyze IT operations. The report finds that 38 percent of the IT leaders surveyed rely on a third-party applications versus 14 percent that have developed their own applications. Another 22 percent collect data manually using spreadsheets, while 20 percent say they don’t measure anything at all. Implementing IT operation analytics requires significant expertise. The report notes that 93 percent of respondents viewed IT analytics as being “critical”, “very important”, or “important.” But only a little over half of the survey respondents (52 percent) are relying on a third-party applications or have built their own applications. That suggests that there’s a major gap that could be filled by managed service providers (MSPs).
or over a decade, a common refrain in the corporate world was that data was exploding and companies needed the capacity to capture that data. As the challenges of data capture have been more or less mitigated, the next challenge has become how to get value out of that data. Between 2015 and the end of 2017, the use of big data analytics grew nearly 40 percent among companies surveyed, according to Dresner Advisory Services, while big data software and services revenues will reach $42 billion this year, and more than $100 billion by 2027. Today, an ever-growing number of solutions are creating opportunities to utilize data for business, science, and technology breakthroughs in ways we’d never imagined. And if you look closely, you’d see that in no place is that more evident than in Ohio, with its growing big data sector.
Allaa "Ella" Hilal is among that rare breed of computer experts who straddle the academic and commercial worlds. As director of data at Ottawa-based Shopify, Hilal oversees data product development for the e-commerce company's international and larger merchants, also known as Plus customers. She is also an adjunct associate professor in the Centre for Pattern Analysis and Machine Intelligence at the University of Waterloo in Ontario, where she earned a Ph.D. in electrical and computer engineering. An expert in data intelligence, wireless sensor networks and autonomous systems, Hilal is among the featured speakers at the Real Business Intelligence Conference on June 27 to 28 in Cambridge, Mass. Here, Hilal discusses what's driving business intelligence (BI) innovation today and some of the pitfalls companies should be aware of.
Executive management is the leading driver of business intelligence in organizations, according to the 2018 Wisdom of Crowds Business Intelligence Market Study recently released by consulting firm Dresner Advisory Services. The annual report is a broad assessment of the BI market, providing a comprehensive look at key user trends, attitudes, and intentions. For each study, users contribute their opinion on topics related to their current and planned use of BI and are asked to prioritize technologies and initiatives strategic to BI.
“The Wisdom of Crowds BI Market Study is the cornerstone of our annual research agenda, providing the most in-depth and data-rich portrait of the state of the BI market,” said Howard Dresner, founder and chief research officer at Dresner Advisory Services. “Drawn from the first-person perspective of users throughout all industries, geographies, and organization sizes, who are involved in varying aspects of BI projects, our report provides a unique look at the drivers of and success with BI.” Survey respondents include IT (28%), followed by Executive Management (22%), and Finance (19%). Sales/Marketing (8%) and the Business Intelligence Competency Center (BICC) (7%). Please see page 15 of the study for specifics on the methodology.
When it comes to business intelligence, Howard Dresner is the expert to speak with. Dresner is credited with coining the term ‘business intelligence’, and has been a thought leader in the BI field ever since. His firm, Dresner Advisory Services, has recently published its 7th annual report on the topic, “2018 Cloud Computing and Business Intelligence Market Study." At 102 pages, the report is the deepest of deep dives into the rapidly growing Cloud BI sector. In a wide ranging conversation, I spoke with Dresner about what the report indicates for the Cloud BI sector, now and in the future. The following interview has been edited and condensed.
Business intelligence (BI) applications have been moving steadily to the cloud for some time now. A new “2018 Cloud Computing and Business Intelligence Market Study” published by Dresner Advisory Services finds the number of organizations planning to increase spending on BI applications running on a public or private cloud to be roughly even at 36 and 37 percent, respectively. But, there are nuances to how organizations want to consume those applications. The top three methods are try and buy, subscription, and as a managed service. All three rank equally in terms of overall importance. When it comes to licensing BI software as a managed service, however, competition between vendors and third-party managed service providers appears to be heating up. The survey finds that while 40 percent of respondents have no preference, 37 percent prefer the managed service to be provided by the vendor. That compares to 23 percent favoring a third-party provider. Not surprisingly, the survey also finds that organizations relying on a third-party vendor prefer the Amazon Web Services (AWS) cloud, which has a slight edge over Microsoft Azure platform, followed by Google Cloud Platform (GCP) and IBM Bluemix.
Cloud BI adoption is soaring in 2018, nearly doubling 2016 adoption levels. Over 90% of Sales & Marketing teams say that Cloud BI is essential for getting their work done in 2018, leading all categories in the survey. 66% of organizations that consider themselves completely successful with Business Intelligence (BI) initiatives currently use the cloud. Financial Services (62%), Technology (54%), and Education (54%) have the highest Cloud BI adoption rates in 2018. 86% of Cloud BI adopters name Amazon AWS as their first choice, 82% name Microsoft Azure, 66% name Google Cloud, and 36% identify IBM Bluemix as their preferred provider of cloud BI services. These and other many other fascinating insights are from Dresner Advisory Services 2018 Cloud Computing and Business Intelligence Market Study (client access reqd.) of the Wisdom of Crowds® series of research. The goal of the 7th annual edition of the study seeks to quantify end-user deployment trends and attitudes toward cloud computing and business intelligence (BI), defined as the technologies, tools, and solutions that employ one or more cloud deployment models. Dresner Advisory Services defines the scope of Business Intelligence (BI) tools and technologies to include query and reporting, OLAP (online analytical processing), data mining and advanced analytics, end-user tools for ad hoc query and analysis, and dashboards for performance monitoring. Please see page 10 of the study for the methodology. The study found the primary barriers to greater cloud BI adoption are enterprises’ concerns regarding data privacy and security.
Many excellent insights are from Dresner Advisory Services’ 2018 Location Intelligence Market Study, part of the Wisdom of Crowds® series of research. Location Intelligence is a form of Business Intelligence (BI) where the dominant dimension of analysis is geography or location with data points overlayed on an interactive map interface. Please see page 11 of the study for the methodology. What makes this report noteworthy is the depth of analysis of the key dynamics, current and future state of Location Intelligence in 2018 and beyond. With the growth of visualization and the rapid emergence of the Internet of Things (IoT) technologies, Location Intelligence is becoming integral to new business models and approaches to gaining insights not accessible before.
The Global Market for IoT Networking Solutions will Grow from $392.1 Billion in 2017 to $1.0 Trillion by 2022 with a CAGR of 21.6% for the Period of 2017-2022, according to a latest study by Research and Markets. IoT advocates show the most interest in initiatives including location intelligence, streaming data analysis, and cognitive BI as well. Also, Business Intelligence Competency Centers (BICC), R&D, Marketing & Sales and Strategic Planning are most likely to see the importance of IoT, said a study by Dresner
Big data adoption reached 53% in 2017 for all companies interviewed, up from 17% in 2015, with telecom and financial services leading early adopters. Reporting, dashboards, advanced visualization end-user “self-service” and data warehousing are the top five technologies and initiatives strategic to business intelligence. Data warehouse optimization remains the top use case for big data, followed by customer/social analysis and predictive maintenance. Among big data distributions, Cloudera is the most popular, followed by Hortonworks, MAP/R, and Amazon EMR. These and many other insights are from Dresner Advisory Services’ insightful 2017 Big Data Analytics Market Study (94 pp., PDF, client accessed reqd), which is part of their Wisdom of Crowds® series of research. This 3rd annual report examines end-user trends and intentions surrounding big data analytics, defined as systems that enable end-user access to and analysis of data contained and managed within the Hadoop ecosystem. The 2017 Big Data Analytics Market Study represents a cross-section of data that spans geographies, functions, organization size, and vertical industries. Please see page 10 of the study for additional details regarding the methodology. “Across the three years of our comprehensive study of big data analytics, we see a significant increase in uptake in usage and a large drop of those with no plans to adopt,” said Howard Dresner, founder and chief research officer at Dresner Advisory Services. “In 2017, IT has emerged as the most typical adopter of big data, although all departments – including finance – are considering future use. This is an indication that big data is becoming less an experimental endeavor and more of a practical pursuit within organizations.”
IoT advocates show the most interest in initiatives including location intelligence, streaming data analysis, and cognitive BI. Also, Business Intelligence Competency Centers (BICC), R&D, Marketing & Sales and Strategic Planning are most likely to see the importance of IoT, said a study by Dresner Advisory Services. Dresner Advisory Services published the 2017 Internet of Things (IoT) Intelligence Market Study, part of the Wisdom of Crowds series of research, where IoT is defined as the network of physical objects, or “things,” embedded with electronics, software, sensors, and connectivity to enable objects to collect and exchange data. The study examines key related technologies such as location intelligence, end-user data preparation, cloud computing, advanced and predictive analytics, and big data analytics.
Manufacturing, Consulting, Business Services and Distribution & Logistics are the top four industries leading IoT adoption. Growing revenue and increasing competitive advantage are the highest priority Business Intelligence (BI) objectives IoT advocates or early adopters are pursuing today. Location intelligence, streaming data analysis, and cognitive BI are the top three most valuable IoT use cases. The higher the BI adoption, the greater the probability of success with IoT initiatives. 53% of all respondents say that IoT is somewhat important with fewer than 15% saying it is critical or very important today. These and many other insights are from Dresner Advisory Services’ 2017 Edition IoT Intelligence Wisdom of Crowds Series study. The study defines IoT as the network of physical objects, or "things," embedded with electronics, software, sensors, and connectivity to enable objects to collect and exchange data. The study examines key related technologies such as location intelligence, end-user data preparation, cloud computing, advanced and predictive analytics, and big data analytics. Please see page 11 of the study for details regarding the methodology. For an excellent overview of Internet of Things (IoT) predictions for 2018, please see Gil Press' post, 10 Predictions For The Internet Of Things (IoT) In 2018.
Dresner Advisory Services recently published the 2017 Small and Mid-Sized Enterprise Business Intelligence Market Study, part of the Wisdom of Crowds® series of research and the 5th annual edition of the study. Dresner Advisory Services defines a small enterprise as an organization with between one and 100 employees; mid-size enterprises have between 101 and 1,000 employees, and large enterprises have more than 1,000 employees. Operations and Management are the greatest Business Intelligence (BI) drivers in organizations of any size as their focus is on improving operational efficiency. Small and mid-size enterprises are more likely to be driven by sales and marketing business roles. The smaller the company, the higher the priority for growing revenues and finding increased competitive advantages using analytics and BI.
The importance placed by organizations on advanced and predictive analytics has slightly declined over the past two years, according to a new study from Dresden Advisory Services. Despite that, APA still remains an important topic to the majority of firms. Participants in the “Advanced and Predictive Analytics Market Study” expressed interest in a broad range of feature requirements, with “regression models, textbook statistic functions and clustering seen as the most important user features/functions,” the study says. “The greatest early adopters of advanced and predictive analytics within the organization are BI experts, business analysts and statisticians and data scientists.”
Reporting, dashboards, advanced visualization and end-user self-service are given the highest priority initiatives in enterprises focused on making BI a strategic foundation for growth. In-memory analytics, in-database analytics, and In-Hadoop analytics are considered the three most critical predictive analytics and BI platforms in enterprises today. Relying on an abundance of predictive analytics features doesn't necessarily lead to a successful BI strategy enterprise wide, but focusing on business outcomes does. These and many other insights are from 2017 Advanced and Predictive Analytics Market Study (PDF, 90 pp., client access) published today by Dresner Advisory Services. The report is available to non-clients here. The study is based on insights gained from interviews with Dresner Advisory Service’s research community of over 3,000 organizations, in addition to vendor customer community interviews. 57% of respondents are from North America, 31% from Europe, the Middle East & Africa, with the remaining 12% from Asia-Pacific (8%) and Latin America (4%). For additional details regarding the methodology, please see page 11 of the study.
BI dashboards are still a top priority for most businesses, but emerging trends around big data are forcing BI and IT teams to rethink how dashboards can be built and used most effectively. Despite the hype surrounding many other analytics technologies, businesses today continue to find the most value in using business intelligence and reporting dashboards. "A lot of people discount reporting and dashboards, but you shouldn't because they're the mainstay of BI," said Howard Dresner, founder, president and chief researcher at analyst firm Dresner Advisory Services LLC in Nashua, N.H.. In a presentation at his company's Real Business Intelligence conference, held at MIT in July, Dresner compared users' perceptions of the value of data dashboards with other analytics technologies based on end-user surveys conducted by the firm. He said big data has only recently crossed the threshold of providing more value than hype.
Despite the current notion that business intelligence and artificial intelligence are separate disciplines, AI-powered BI is an idea that should get more attention. Over the past month, I spent time at conferences dedicated to both poles of the analytics spectrum: business intelligence and artificial intelligence. I came away wondering why they're so far apart. Both involve number crunching at their core. But while BI is primarily retrospective in nature, AI is all about the future. The statistical analysis behind BI is primarily about basic counts, in contrast with the ultra-sophisticated machine learning and deep learning algorithms that underpin AI. Because of these differences, and the fact that AI as a useful tool is relatively new while BI is yesterday's news, we treat the two disciplines as wholly separate. During my time at the O'Reilly AI Conference in New York in late June, I heard not a single mention of BI. Conversely, at Dresner Advisory Services LLC's Real Business Intelligence conference held on the campus of MIT in July, AI was primarily used as a counterpoint to BI in discussions of the latter's strong business value and high adoption.
Many businesses commit similar mistakes when creating BI reports. Consultant Mico Yuk offers tips to help you avoid some of the most common pitfalls. By now, just about every company has some kind of BI reporting system in place. And just about every company is doing reporting wrong. "We're making a lot of pretty garbage," BI consultant Mico Yuk said in a presentation at Dresner Advisory Services LLC's Real Business Intelligence conference, held in Cambridge, Mass., in July 2017. Yuk, CEO at consulting group BI Brainz, said most companies have problems in their BI reports that diminish their value. The BI world has spent the last few years focusing on making reports visually compelling and easy to create, but there's a lot more to a good report than how it looks, she cautioned.
Howard Dresner, widely credited with coining the term BI, has some advice for companies determined to take their business intelligence programs to the next level: Consider the source. Dresner is founder and chief research officer at Dresner Advisory Services LLC in Nashua, N.H. He believes a lot of the hype around technologies such as machine learning should be taken with a big grain of salt. Dresner, who is hosting the Real Business Intelligence Conference in Cambridge, Mass., on July 11 and 12, sat down with SearchCIO to talk about how the BI market has matured, what BI myths need busting and metrics that address the perennial problem of getting IT and the business on the same page. Below are excerpts from the interview; click on the player to hear my interview with Howard Dresner in its entirety.
Running business intelligence applications in the cloud has yet to take the BI world by storm. Researcher Howard Dresner offers insight on why and discusses the state of cloud BI software. Cloud-based business intelligence has been a slowly simmering part of the BI market for more than 10 years, and researcher Howard Dresner says that doesn't appear to be changing significantly, despite the overall growth of cloud computing. An annual survey on cloud BI usage conducted by Dresner's company, Dresner Advisory Services LLC, did find a small uptick in adoption this year. Thirty-one percent of the 383 respondents from user organizations said their companies are using cloud BI tools, up from 25% in 2016. But there was a nearly corresponding drop in the percentage of respondents saying they might deploy cloud-based BI technology in the future. Meanwhile, the percentage of respondents with no intention of going to the cloud for BI held steady at 38%, according to a recently published report on the survey.
Companies and customers don't spend much time thinking about the negative side effects of data science, but they may want to start. In her new book, Weapons of Math Destruction, Cathy O'Neil explains that just because an algorithm exists, it doesn't mean it's a good algorithm. Many of the algorithms that control our lives are flawed and need to be debugged, like any software. O'Neil, a data scientist, isn't talking about the algorithms that are designed to trick people, such as the Volkswagen emissions algorithm. Instead, she's talking about algorithms that may lead businesses and governments to draw erroneous, biased and even harmful conclusions about customers and constituents. O'Neil, who will be speaking at the upcoming Real Business Intelligence Conference in Cambridge, Mass., sat down with SearchCIO to talk about why algorithms go bad. This Q&A has been edited for brevity and clarity.
The expectation these days is that data science will deliver smarter, fairer, less biased and more consistent decisions. But according to Cathy O'Neil, data scientist and author of the recently published Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy, it will be a while before data science makes good on that promise. O'Neil, a data skeptic, likens data science to the early days of the automobile industry, before drivers knew to question the fallibility of cars and before safety standards were established. Today, O'Neil said businesses appear to trust mathematical algorithms without question, putting blind faith in algorithms that may rely on immoral and possibly illegal methods. She said she believes it is time to establish a national regulatory board to ensure algorithms do less harm -- an idea she borrowed from Ben Shneiderman's recent talk at the Alan Turing Institute.
A growing number of organizations are turning to the cloud for business intelligence applications, with private clouds narrowly surpassing public clouds as the platform of choice. There was a 6 percent overall increase in organizations implementing cloud-based business intelligence applications in 2016, according to Dresner Advisory Services, which has just released the "2017 Cloud Computing and Business Intelligence Market Study." The study reveals that ad hoc query, advanced visualization, dashboards, data integration and data quality, end-user self-service and reporting are the most required cloud business intelligence features. “Adoption continues to increase year-over-year, especially for public cloud offerings,” says Howard Dresner, founder and chief research officer at Dresner Advisory Services. “This has been driven largely by smaller organizations but has become more widespread.” According to Dresner, “for the most part, this year’s results builds upon previous years’: importance [of business intelligence applications in the cloud] decreased and adoption increased. Many of those that said they weren’t considering cloud-based solutions last year or two years ago are now considering or have already adopted.”
Earlier this week Dresner Advisory Services published their 6th annual report on Cloud Business Intelligence (BI) titled 2017 Cloud Computing and Business Intelligence Market Study (101 pp., PDF, client access reqd). This report provides insights into deployment trends & attitudes toward cloud business intelligence (BI) defined as the technologies, tools, and solutions that employ one or more cloud deployment models. “We began to analyze this market dynamic back in 2012 when adoption was nascent, and since that time we have seen deployments of public cloud BI applications continue to grow steadily,” said Howard Dresner, founder and chief research officer at Dresner Advisory Services. “Organizations are citing substantial benefits over traditional on-premises implementations. While challenges remain and improvements can certainly be made, we believe that cloud business intelligence, particularly public cloud offerings, will continue to move into the mainstream.”
Without effective, governed self-service data preparation and data discovery, information becomes noise, trust in the information is diminished, and effective collaboration becomes much more difficult. Everything is louder than everything else. Self-service business intelligence platforms provide significant benefits, but have also contributed to a new trend: the "wild west" of proliferating BI silos, inconsistent business definitions, no data lineage, and no single version of the truth. But can the "need for speed" in business decision-making be reconciled with the need for governance? According to Howard Dresner, governance of BI content creation and sharing correlates strongly to success with BI, improving information consistency and accelerating group-based decision making.
Research and Development and executive management are two functions that place the greatest importance on location intelligence analytics and business intelligence (BI). Energy and transportation industries place the most significant emphasis on location intelligence analytics and BI today. 63% of respondents perceive location intelligence as being critically (20%) and very important (43%) to their ongoing business operations. Executive management is most interested in cloud-based deployments of location intelligence analytics, and BI gives cloud its highest criticality ranking. These and many other insights are from an excellent research study recently published by Dresner Advisory Services titled 2017 Location Intelligence Market Study Report (92 pp., PDF, client access). Dresner Advisory Services defines location intelligence as a form of business intelligence where the dominant dimension used for analysis is location or geography. Most typically, though not exclusively, analyses are conducted by viewing data points overlaid onto an interactive map interface.
Mobile BI apps may have lost some of their luster from earlier years, but PetSmart, for one, continues to find real business value in apps for their managers and executives. Who needs offices when, thanks in large part to the maturity of mobile business intelligence applications, insights can be delivered to managers wherever they find themselves? That was a realization that PetSmart Inc. recently arrived at for its main distribution center. The international pet supply store has been using mobile BI apps since 2010. During that time, the company has deployed nearly 20 different apps for everything from executive reporting to supply chain monitoring.
Big data is still too difficult. Despite all the hype—and there has been lots and lots of hype—most enterprises still struggle to get value from their data. This led Dresner Advisory Services to conclude, "Despite an extended period of awareness building and hype, actual deployment of big data analytics is not broadly applicable to most organizations at the present time." Ouch. Some of this is a people problem. However persuasive the data, executives often prefer to ignore that data. But, a big part of the complexity in big data is about the software required to grok it all. Though Spark and other, newer systems have improved the trajectory, big data infrastructure remains way too hard.
Dresner Advisory Services has released its 2016 Internet of Things and Business Intelligence Market Study as part of its 'Wisdom of Crowds' research series and has found, among other insights, that IoT remains a primary large-organisation phenomenon. The second annual report puts forth an understanding of current demand for and interest in business intelligence (BI) in IoT, and examines related technologies such as end-user data preparation, location intelligence, advanced and predictive analytics, big data analytics and cloud computing. The report shows that mature BI programmes are stronger IoT advocates, with greater success and strong promotion and expansion of business intelligence. IoT advocates have a significantly high interest in big data analytics, with a three times higher likelihood of considering big data as 'critical'.
Few companies are using advanced analytics, despite experts' bullish outlook on its potential as a business game changer. How can you get started? Advanced and predictive analytics seems to be at the top of its game, at least when it comes to hype. Ninety percent of companies rated advanced and predictive analytics at some level of importance in the 2016 Advanced and Predictive Analytics Market Study, which is conducted annually by Dresner Advisory Services. It lands at number six on the firm's list of 30 top tech trends, ahead of cloud, governance and even Big Data. Here's the rub: The survey found the adoption rate for advanced and predictive analytics was 24 percent. Howard Dresner, chief research officer for Dresner Advisory Services, believes there's little that indicates the number will grow much will in the near future.
- Sales and strategic planning teams see IoT as the most valuable. - IoT advocates are 3X as likely to consider big data critical to the success of their initiatives & programs. - Amazon and Cloudera are the highest ranked big data distributions followed by Hortonworks and Map/R. - Apache Spark MLib is the most known technology on the nascent machine learning landscape today. These and many other excellent insights are from Dresner Advisory Services’ 2016 The Internet of Things and Business Intelligence Market Study published last month. What makes this study noteworthy is the depth of analysis and insights the Dresner analyst team delivers regarding the intersection of big data and the Internet of Things (IoT), big data adoption, analytics, and big data distributions. The report also provides an analysis of Cloud Business Intelligence (BI) feature requirements, architecture, and security insights. IoT adoption is thoroughly covered in the study, with a key finding being that large organizations or enterprises are the strongest catalyst of IoT adoption and use. Mature BI programs are also strong advocates or adopters of IoT and as a result experience greater BI success. IoT advocates are defined as those respondents that rated IoT as either critical or very important to their initiatives and strategies.
It used to be that only well-funded corporate types had access to business intelligence (BI) and data analytics tools. By helping to turn mountains of impenetrable data into insightful and actionable business information, these tools allow large businesses to monitor the performance of their internal business processes, to spot customer trends, and to adjust their strategies accordingly. Today, small business owners can get in on the act, thanks in large part to cloud-based BI software. Affordable and user-friendly, the current crop of BI products "helps SMEs [small and midsized enterprises] level the playing field," Howard Dresner, founder and chief research officer at Dresner Advisory Services, told Small Business Computing. Whereas the corner shop may have been "limited to Excel" to make sense of its data in the past, the latest breed of BI tools for small business are "far more sophisticated," and they can deliver many of the same insights that help large enterprises remain adaptable to changing market conditions.
When it comes to the selection of an analytics tool for a new Business Intelligence initiative, where should you start? A typical internet search may yield some worthwhile answers, though you’ll likely be bombarded with sponsored links to so-called ‘expert’ analysts pushing their own agendas. Consulting trustworthy sources of information is the name of the game. At Solutions Review, we put ourselves in the middle of it all, constantly coming across resources that can assist buyers of enterprise technology to achieve their goals in selecting the tools that best fit their needs. With this in mind, we’ve compiled a list of the 10 best resources solutions-seekers should consult while in the research phase of a new analytics project. Since each organization needs a tool to match up with their own specific use cases, these resources allow researchers to sift through solutions that satisfy a wide range of BI requirement
Embedded business intelligence software lets companies build BI functionality into other types of applications so end users can access reports and analyze data without having to open up a separate tool. Embedded BI "isn't a barn burner" of a technology, according to analyst Howard Dresner -- but it has a role to play in making BI data easier to use in business operations. Dresner, founder and chief research officer at Dresner Advisory Services LLC in Nashua, N.H., discussed the current status and future potential of embedded BI tools in a Q&A with SearchBusinessAnalytics. The discussion was based partly on the results of his company's 2016 Wisdom of Crowds survey, which canvassed more than 1,500 IT, BI and business professionals on the BI and analytics priorities in their organizations. Excerpts from the interview follow.
Embedded BI software is helping organizations add value to applications and monetize their growing data stores, without requiring end users to turn to separate analytics tools. Embedded business intelligence (BI) and reporting is a popular option for companies like Urban Airship that are looking to increase the value of existing applications or monetize their data by creating new products and services built around analytics capabilities. Close to 70% of the 1,524 respondents to a survey conducted this year by research firm Dresner Advisory Services LLC said embedded BI was either important, very important or critical to their business. That ranked embedded BI 12th for strategic importance out of 30 analytics technologies the respondents were asked about, according to Dresner's 2016 Wisdom of Crowds Business Intelligence Market Study report.
A recalibration is taking place in the way that companies staff and organize for success with business intelligence. We discussed the topic of organizing for success in one of my recent weekly #BIWisdom tweetchat sessions on Twitter. Leading into the discussion, I tweeted that success requires the right culture and strong C-level management to drive alignment of all involved parties to make BI work at a strategic level. Also required is an organizational model and set of processes, methods, etc. And well-planned success criteria and metrics are key. It’s also useful to have a funding model and some potential projects lined up. And it’s necessary to take inventory of resources, identify skills and knowledge gaps and and plan to augment, when necessary, with external resources. And, of course, processing and preparing data is important; but where, when and how that takes place will be determined by these other questions.
A data-driven culture and the tools to support that environment – including self-service – deliver superior results, indicates a trio of recent surveys. There’s a continuing role for IT, too. While no one seriously disputes the value of business intelligence/analytics in business, government, education or other nonprofit organizations, the drivers of return on investment appear to be evolving. Comparing the results of three recent surveys to a 2006 poll, I found a few perennial drivers of analytics RoI/value, but other technologies or initiatives have become much more important.
“How you gather, manage and use information will determine whether you win or lose,” stated Bill Gates. That’s a lot of what we track in our annual Dresner Advisory Services Wisdom of Crowds(R) Business Intelligence Market Study. And attendees’ attention was sharp at one of my weekly #BIWisdom tweetchat sessions where I revealed some findings of our 2016 report before it was released to the public. One of the significant findings of the study is that user adoption or penetration (meaning the number of BI users as a percentage of the overall population of an organization) grew noticeably in all sizes of organizations from 2015 to 2016. The level of penetration has increased annually, but it was noticeably stronger this year than from 2014 to 2015.
Even as advanced business analytics tools, such as cognitive computing, machine learning and internet of things analytics, grow in prominence, businesses still derive the most value from good, old-fashioned BI reporting, according to a new report from Dresner Advisory Services. The report, the 2016 Wisdom of Crowds Business Intelligence Market Study, ranked reporting, dashboards and self-service as the top three functions businesses look for in BI and analytics tools. Hotter topics, like internet of things (IoT), social media analysis, cognitive BI and edge computing, all ranked in the bottom five on the list of functions businesses most want. Hot big data tools, like Hadoop and streaming data analysis, were in the bottom third.
This week, we have a special treat for you! On this episode of Analytics on Fire, we’re featuring an exclusive interview with the man who coined the term “Business Intelligence” in 1989 – Howard Dresner. Today’s topic is not only special and insightful, but it is also extremely new. Howard just recently coined another term – “Collective Insights.”
Five years ago, when my company began conducting annual market studies of the use of the cloud computing model in business intelligence (BI) deployments, the model’s future was “cloudy” in the BI world. Each year our survey examines the “importance” that users associate with cloud BI. Now with five years of data tracking the market, it’s clear that there is a real sea change in attitudes toward cloud BI and it’s entering new territory. Survey respondents’ ranking of importance peaked in 2014 and has declined since then. However, as the figure below shows, only 38 percent of respondents in our 2016 study report they have no plans for cloud BI. So while the “importance” of cloud BI adoption may be declining, deployments are actually increasing. This is a significant finding: it shows real progress and is a sign of a healthy, maturing market. In maturing technology segments, it’s not unusual for the sense of urgency around adoption to recede from initial market hype levels even while deployments/adoption increase.
When it comes to the selection of an analytics tool for a new Business Intelligence initiative, where should you start? A typical internet search may yield some worthwhile answers, though you’ll likely be bombarded with sponsored links to so-called ‘expert’ analysts pushing their own agendas. Consulting trustworthy sources of information is the name of the game. With this in mind, we’ve compiled a list of the top resources BI solution-seekers should consult while in the research phase of a new project. Since each organization needs a tool to match up with their own specific use cases, these resources allow researchers to sift through solutions that satisfy a wide range of BI requirements. Dresner Advisory Services Wisdom of Crowds Dresner Advisory Services is an independent analyst firm that specializes in Business Intelligence and related markets. The firm’s founder, Howard Dresner, is one of the foremost thought leaders in the world of enterprise Business Intelligence, he even coined the term in 1989. Howard Dresner spent 13 years at Gartner where he acted as VP, Research Fellow and Team Leader for the BI space, having been responsible for Gartner’s patented Magic Quadrants during his tenure. Dresner does not pre-fund their market studies, meaning that they take no sponsor money. This means that you can be absolutely sure that Dresner studies are agnostic and reflect only the industry at large. The Wisdom of Crowds studies leave no stone unturned, and cover every avenue of the BI market, including Cloud Computing and Business Intelligence, Big Data Analytics, Embedded Business Intelligence, SMB Enterprise Business Intelligence, Advanced and Predictive Analytics, and many more.
Users weaned on Excel want more robust data preparation capabilities, finds a Dresner Advisory Services study. Because data preparation has largely been the responsibility of time-crunched IT organizations, it often creates a bottleneck in data analytics. The problem has gotten lots of attention the past few years, with vendors rolling out products that aim to help business people employ even complex technologies like Hadoop. But it all started with Microsoft Excel, said Howard Dresner, founder and president of Dresner Advisory Services, the humble office productivity tool that put basic data preparation abilities into the hands of hordes of business people. Use of Excel has “created a pent-up demand for something better,” he said, noting that “Excel is not especially good at data preparation. It presents data in a table format, you can do find and replace, but it is a general purpose tool. It was not designed to be a data preparation tool, even though that is how many of us use it.”
Interest is growing in location intelligence, but the technology isn’t a priority for most businesses, according to the third annual Location Intelligence Market Study published last month by Dresner Advisory Services LLC. Location intelligence refers to a business intelligence tool that relates geographic information from a variety of data sources, including GIS and aerial maps, to business data. Respondents to the survey, which included 403 industry representatives from technology to health care to financial services, ranked location intelligence/analytics 12th out of 25 technologies. Respondents pointed to dashboards, data discovery, data mining and integration with operational processes as bigger priorities than location intelligence; they ranked topics such as in-memory analysis, big data, text analytics and the Internet of Things (IoT) as lesser priorities than location intelligence. The findings don’t surprise Howard Dresner, founder and chief research officer at Dresner Advisory Services. Interest in location intelligence is dependent on the industry. “If you’re doing things like sales operational planning, you have to use location intelligence to do that. Otherwise, you’re not going to understand how to allocate resources appropriately,” he said.
Big data is perhaps the most hyped term in the tech circle today. Not only tech enthusiasts, but research firms also are upbeat on big data and how it would help transform modern transformations. Many believe big data is now mainstream. However, a research report from Dresner Advisory Services adds a sober note to the Big Data hype. Dresner is helmed by Howard Dresner, president, founder and chief research officer at Dresner Advisory Services, LLC, an independent advisory firm. Dresner’s research report, from November 2015, includes input from roughly 3,000 organizations globally, as well as crowdsourcing and vendors’ customer communities. The study analyzes user perceptions and intentions around big data analytics, systems that enable end-user access to and analysis of data contained and managed within the Hadoop ecosystem.
Big Data gets a lot of headlines. If any technology can be called heavily hyped, Big Data earns the prize for most breathless predictions of enterprise influence. Typical of the rosy predictions is this from IDC: spending on Big Data-related infrastructure, software and services will grow at a torrid compound annual rate of 23.1 percent between 2014 and 2019, reaching a hefty $48.6 billion in 2019. For companies, it would seem, boosting revenue as easy as implementing a Big Data solution and hiring an accountant who can track your windfall profits. However, a research report from Dresner Advisory Services adds a sober note to the Big Data hype. Dresner is helmed by Howard Dresner, who understands the market: he’s widely known as “the father of business intelligence” (having coined the term). He led Gartner’s business intelligence research practice for 13 years. Dresner’s research report, from November 2015, includes input from roughly 3,000 organizations, as well as crowdsourcing and vendors’ customer communities. The survey reports that: A mere 17 percent actively use Big Data in their organization today. A lukewarm 47 percent “may use” Big Data in the future. A remarkably large 36 percent have no plans for Big Data. In essence, fully a third of companies say: Big Data, who cares?
A new report from Dresner Advisory Services says businesses see increased value in location intelligence software, but products need to improve. Businesses increasingly see location intelligence software as central to their corporate initiatives, but vendor products still have a ways to go to match users’ preferences, according to a new report from Dresner Advisory Services. In the survey of IT and business users, respondents ranked location-mapping software as their 12th most important business intelligence-related technology initiative — behind mainstays of the field, such as dashboard development, self-service BI and data warehousing, but ahead of emerging technologies like cloud BI, big data and the Internet of Things.
Two separate studies recently confirmed that the Big Data skills shortage is still hampering enterprises, most of which have yet to put the technology to use. While there’s no direct cause-and-effect linkage, the A.T. Kearney report would seem to support the conclusion of another new study that shows a persistent low adoption rate of Big Data technologies in the enterprise. That report, “Big Data Analytics Market Study,” was published a couple weeks ago by Dresner Advisory Services LLC. While it costs $595 to read, a promotional teaser indicates few companies have yet to hop onto the Big Data bandwagon. “According to the study, current adoption is low with only 17 percent of respondents actively using Big Data in their organization today and just under 50 percent indicating they may adopt Big Data in the future,” the company said. “Big Data analytics is ranked in the bottom third of 25 strategic business intelligence (BI) initiatives currently under study in 2015.”
When it comes to big data, vendors are saying one thing, and organizations are doing something very different. According to the first edition of Dresner Advisory Services’ Big Data Analytics Market Study, the big data industry may talk up the importance of big data, but customers don’t have a lot of plans to adopt it — at least, not in the form that Hadoop currently presents. Hadoop? Maybe someday Around 3,000 organizations were polled, and only 17 percent “actively use big data in their organization today,” with big data meaning Hadoop; 47 percent “may adopt big data in the future.” That leaves 36 percent who have no professed plans for big data — and 69 percent of potential future adopters aren’t even thinking about doing anything about it until after 2016.
Before you dismiss this blog because you think whether or not to outsource is an old question long settled in many businesses, consider this: outsourcing is making a dent in the BI environment. I’m talking about outsourcing analysis and generating insight, not just admin functions. The driver in many cases is the lack of in-house data scientists and/or analytical skills. Organizations believe both can be a critical distinction for success. I posed the “O” word as a topic in one of my recent #BIWisdom tweetchats, as the tribe has been privy to outsourcing deals from every angle (as the buyer, the vendor, the consultant) and definitely have opinions.
Back in March, we listed the top ten big data Twitter follows for IT pros in order to turn your feed into a veritable treasure trove of insightful big data news and information, and then a few months later we gave you the five best accounts to follow for data integration insights . We hope that our suggestions helped you achieve a bit of analytical enlightenment, but we recognize that Twitter is a big, big place. Twitter’s active user base is 243 million users; nearly four times the population of the United Kingdom. It would be foolhardy to think that we could get all of the best enterprise technology accounts in a brief list. So we’re back at it again, this time helping you inundate your Twitter feed with the best business intelligence news, thought leadership, and best practices in what’s becoming an ongoing series. Without further ado, here are the top ten business intelligence Twitter influencers. There are no companies, journals or news sites included here; just the most insightful and passionate professionals in the industry.