Facebook Twitter LinkedIn YouTube E-mail

Deep Learning for Intelligent Commerce

Share

Author: Matthew Simonson

Artificial intelligence is a topic of intense media hype. Machine learning, deep learning, and AI come up in countless articles, with deep learning being heralded as an incredible breakthrough in AI. In this blog post I will discuss deep learning, what deep learning has achieved so far, the significance of these contributions, how deep learning extends the capabilities of “shallow” machine learning approaches, and why deep learning is the “right” approach for companies to be investing in and for researchers to flock to.

What is machine learning?

Before we discuss deep learning I think it’s important to have some understanding of what machine learning is, and how deep learning relates. Machine learning arises from this question: can a computer learn how to perform some task without having been given the step-by-step instructions for completion? Rather than programmers crafting data-processing rules by hand, could a computer automatically learn these rules by looking at data? In classical programming, humans input rules (a program) and data to be processed according to these rules. If the program works, we get a result as output. With machine learning, humans input example data as well as example answers expected from the data, and our output is a set of rules that can be used to generate the desired data output. These rules can then be applied to new data to produce original answers. A machine-learning system is trained rather than explicitly programmed. A machine-learning model transforms its input data into meaningful output using a process that is “learned” from exposure to known examples of inputs and outputs.

The central problem in machine learning is to meaningfully transform data. Machine-learning models are all about finding appropriate representations for their input data—transformations of the data that make it more amenable to the task at hand. All machine-learning algorithms consist of automatically finding such transformations that turn data into more useful representations for a given task.

Deep learning vs. machine learning

Deep learning is a specific subfield of machine learning and is another approach to take on learning representations from data. With deep learning, the problem of meaningfully transforming data into useful representations is broken down in a step by step manner, learning successive layers of increasingly meaningful representations. The “deep” in deep learning isn’t a reference to any kind of deeper understanding achieved by the approach; rather, it stands for this idea of successive layers of representations. Modern deep learning often involves tens or even hundreds of successive layers of representation that are all learned automatically from exposure to training data. There are two defining characteristics that describe how deep learning learns from data. The first is the incremental, layer-by-layer way in which increasingly complex representations are learned within every subsequent layer, and the second is the fact these intermediate incremental representations are combined and optimized jointly with respect to the models overall performance.

Alternatively, other approaches to machine learning tend to focus on learning only one or a few layers of representations of the data; hence they’re sometimes called shallow learning.

Deep dive into deep learning

So, what is deep learning exactly? Deep learning is a mathematical framework for learning representations from data. In deep learning, layered representations are learned using models called neural networks, structured in literal layers stacked one after the other. You can think of a deep network as a multistage information-distillation operation, where information goes through successive filters and comes out increasingly purified.

What has deep learning achieved so far? Specifically, deep learning has achieved the following breakthroughs in the following areas of machine learning (as well as other not listed here):

    • Near-human-level speech recognition
    • Ability to answer natural-language questions
    • Improved machine translation
    • Improved text-to-speech conversion
    • Near-human-level image classification
    • Near-human-level handwriting transcription
    • Visual art generation (artistic style extraction and transfer)
    • Near-human-level autonomous driving
    • Digital assistants such as Siri, Google Now, and Amazon Alexa
    • Improved advertisement targeting, as used by Google, Baidu, and Bing, and others
    • Deep reinforcement learning has been used to approximate the value of possible direct marketing actions
    • Improved search results on the Web
    • Recommendation systems have used deep learning to extract meaningful features for content-based music recommendations
    • Automated drug discovery by predicting novel candidate biomolecules for disease targets
    • Several applications in bioinformatics
    • Superhuman game playing ability

What makes deep learning different? One reason deep learning is so popular is that it offers better performance on many tasks. Deep learning also makes problem-solving much easier, because it largely automates the process of feature engineering, the most crucial step in a machine-learning workflow. Feature engineering is a time-consuming process where the initial input data is manually transformed into the format required for a given analysis. Alternatively, deep learning largely automates this step: with deep learning, you learn all features in one pass rather than having to engineer them yourself. Have missing values in your data? Not a problem for deep learning, it will learn to ignore them. Have a huge number of features with complex interrelationships? Not a problem for deep learning, it will learn to properly weight these features and untangle their interrelationships. This has greatly simplified machine-learning workflows, often replacing sophisticated multistage pipelines with a single, simple, end-to-end deep-learning model.

Is there anything special about deep neural networks that makes them the “right” approach? Absolutely, some specific examples include:

  • Simplicity: Deep learning reduces and automates much of the feature engineering process, replacing complex, brittle, engineering-heavy pipelines with simple, end-to-end trainable models.
  • Scalability: Deep learning is easily parallelized and can take advantage of high powered graphical processing units (GPUs). Also, deep-learning models are trained by iterating over small batches of data, allowing them to be trained on datasets of arbitrary size.
  • Versatility and reusability: Unlike many prior machine-learning approaches, deep-learning models can be trained on additional data without restarting from scratch, an important property for very large production models. Furthermore, trained deep-learning models are easily repurposed and reused. For example, it’s possible to take a deep-learning model trained for image classification and drop it into a video-processing pipeline. This allows customers to reinvest previous work into increasingly complex and powerful models.

Summary

In this blog post I have reviewed deep learning, what deep learning has achieved so far, the significance of these contributions, how deep learning extends the capabilities of “shallow” machine learning approaches, and when deep learning is the right approach for finding meaningful insights in your data that are hidden beneath the surface, beyond the simple observed trends. Contact Nihilent to find out how you can add deep learning to your company.

 
 Share on Facebook Share on Twitter Share on Reddit Share on LinkedIn
Comments Off on Deep Learning for Intelligent Commerce  comments 

Adding agility to business intelligence delivery

Share

Author: Greg Baldini

The promise of BI

The promise of BI has always been better decisions through data. As BI professionals, we hope to empower business users to make new discoveries, ask better questions, and be able to answer those questions.

How many times do we see BI development end in a flop, though? Have you ever observed the battle between business users and IT? Why is it that the promise of BI always seems to cast a shadow over the delivery?

Methodologies

Waterfall

The waterfall methodology is the traditional approach to BI. Waterfall type development is made up of large, complete products in a single delivery. Also referred to as a “big bang” release, waterfall projects are typified by robust, enterprise-ready deliverables. That is the promise – with solid requirements and specifications, after development and testing, we will deliver complete and functional solutions.

Agile

Agile methodologies focus on end-user engagement, small, functional deliverables, and rapid iteration on those deliverables. Requirements and specifications are tackled in collaboration with end users, who provide continuous feedback throughout the short development cycles. This feedback is then incorporated into the next sprint. That is the promise – with continuous engagement, we will deliver functionality quickly and be responsive to feedback.

Mo’ methodologies, mo’ problems

Both methodologies have promise, and both have drawbacks in BI.

On the business side, users see IT and a waterfall approach as long, slow, and expensive, with an unknown payoff as a kicker. From IT’s perspective, business users have endless and constantly changing needs beyond development capacity. How long is your development backlog?

We face a tradeoff: an agile-only strategy addresses present business needs, whereas a waterfall-only strategy provides a robust enterprise architecture. Can we really do without either?

So how do we add agility?

Traditionally, the gap between these approaches has been filled by business analysts using Excel. Rob Collie observes that “Export to Excel” is the third most commonly used button in any BI tool, after “Okay” and “Cancel.” The analyst will take up the slack in BI development. Little reporting changes turn into huge tracts of business logic tied up in individuals’ Excel workbooks.

Eventually, we arrive in Excel hell. This is a situation I think we all know well.

The way out, perhaps unintuitively, to embrace its cause: business users can understand and discover their data and reporting needs better than IT.

Pick your poisons

We cannot solve the BI problem with either waterfall or agile methodologies alone. We can solve the BI problem with pieces from the waterfall and agile methodologies.

Modern BI tools empower end users to take control of data without relying on IT. Power BI, in particular, offers a powerful set of tools to facilitate collaboration between users and IT.

With Power Query and Power Pivot, an Excel power user can perform ETL and model data for immediate reporting needs. Business-driven development ensures constant feedback and facilitates the iterative approach that is proven so powerful in agile methodologies. Individuals and small teams meet their data needs in the short term with minimal IT involvement using modern self-service BI tools. Additionally, these tools offer a way for the business users to simultaneously work with IT.

The Power BI Service offers a collaboration portal – a sandbox that can be controlled and maintained by IT. It becomes easy to monitor usage and audit published reports.

When it comes time for enterprise development, there is an audit trail of ETL sources and processes. Power Query provides a list of each source and every transformation. Power Pivot enforces a tabular structure and defined measure logic, so can be seamlessly promoted into SQL Server Analysis Services. The final IT products can be exposed transparently in the same Power BI portal – a single pane of glass for all the organization’s data.

The business-driven solution easily feeds into IT’s development cycle, with clear, well-tested requirements and prototype implementations. This is how we create a hybrid agile approach, proven to be extremely powerful in increasing effectiveness of BI programs:

Companies that have complemented their traditional BI solutions with the agile approach are almost 50% more likely to consistently meet the needs of business managers than those that rely solely on traditional BI solutions.

Source: Agile BI: Complementing Traditional BI to Address the Shrinking Decision Window, November 2011, Aberdeen Group

The way we approach BI

Traditional IT waterfall delivery models do not completely translate to BI projects. However, in our experience, pure agile delivery models don’t work well for BI solutions either.

Just like IT projects are promoted dev -> test -> prod, BI solutions should be promoted individual / team -> department -> enterprise. With a hybrid agile approach, we can finally deliver on the promise of BI:

Better decisions through data.

 

If you would like to learn more about improving your business and empowering your end users, please feel free to contact us, or join us at one of our user group meetings. Microsoft BI User Group of Minnesota | Power BI User Group of Philadelphia

 
 Share on Facebook Share on Twitter Share on Reddit Share on LinkedIn
Comments Off on Adding agility to business intelligence delivery  comments 

Your business is dynamic. Your data should be too.

Share

The speed of business is increasing. Executives are working to respond quickly to new buyer behaviors, new market opportunities, new types of competitors threatening to disrupt existing business models, and to harness digital technologies that are evolving rapidly.

As a result, business agility is becoming a core requirement for businesses. Not having the right data, platforms, and processes to enable business agility can be the difference between market leadership and market irrelevance.

What is business agility?

The characteristics of business agility can be simply defined as:

  1. Faster, better decision-making based on real-time data,
  2. Flexible production, staffing, and operational models that can respond to and predict changing conditions quickly,
  3. More fluid ways of working within the enterprise and with external agents such as customers, suppliers, as well as distributed people and resources.

A critical foundation underlying these agile business definitions is the access to real-time and dynamic data that enables these better insights and decisions and improves speed to market. It makes sense that if your business is dynamic, your data should be too.

Making Agile Business intelligence a core component of your BI and Advanced Analytics strategy will allow you to build a foundation with your data. Agile BI enables your analysts and executives to access reporting based on data that is real-time. Your analysts and executives can then continuously apply new business rules and explore what-if scenarios.

How do you create your Agile BI strategy?

It starts with understanding the business strategy and the business leaders’ use cases for the data. Typically, business outcomes that drive financial, operational, or sales improvements are the first areas that enterprises seek to improve through improved business intelligence. Do you have a shared vision for the business outcomes you are trying to achieve? As an example, are you working to create a more dynamic pricing model based on responding to and predicting competitors’ pricing? If the use cases are not initially clear, our team at Nihilent can help you explore the possibilities. First, we’ll look for patterns in your existing data and help you identify gaps in the process, data, and consumption of data that can drive real cross functional value from customer service/CRM to HR to Operations, as an example.

Once the initial use cases are clear, and you have an understanding of the business outcomes the business leader is expected to drive; then, you review the data. Do you have the data? Where are the gaps in data and how do you fill those gaps? It’s also important to know what business rules are in place or need to be created that will drive the decisions taken from the data.

With an Agile BI approach, you do these first steps quickly knowing that the value of Agile BI is based on a process of continuous feedback and rapid iteration.

If you have historical data to leverage, you also don’t need to wait until later phases of your BI project to take advantage of Advanced Analytics and Machine Learning capabilities. They can be part of your BI solution from the beginning enabling you to accelerate the value and ROI that you will achieve from your BI investments.

Ready to get started on your Agile BI initiative? Bring together your team of business sponsors, analysts, IT and a trusted BI partner. Then, accelerate ROI on your BI investments by creating a plan for agile, rapid, iterative BI that adjusts quickly as your business needs change.

Contact us to learn more about our Agile BI Quick Start Program. You may qualify for two free days of BI advisory services.

With Agile BI, you can more easily take advantage of your data assets to monetize your data. Download our Insider’s Guide on Monetizing your Company’s Data to learn how to generate revenue from one of your most important assets.

 
 Share on Facebook Share on Twitter Share on Reddit Share on LinkedIn
Comments Off on Your business is dynamic. Your data should be too.  comments 

Data Science – Harnessing Advanced Analytics for your Business Intelligence Digital Transformation

Share

BLOG BY: Susan Van Riper, GNet Group

No matter what business you are in, Data Science will be a fundamental component in your digital transformation efforts. By looking at patterns in data and implementing workflows, processes, and software that can automatically and reliably turn that data into actionable insight. Data Science can give your business an advantage that is difficult or even impossible for competitors to match.

No matter which path you choose to get started with Data Science, the important point is to get started!

Here are some steps to get you started toward a data-driven culture:

  1. Find a Data Scientist. But, who is the ideal Data Scientist? The ideal Data Scientist has a diverse set of traits and skills:
    1. Statistical analysis and machine learning
    2. Statistical and mathematical tools
    3. Programming and database
    4. Data modeling, warehouse, and unstructured data
    5. Solution deployment architecture
    6. Business domain knowledge
    7. Visualization
    8. Storytelling

Where do you find this Data Scientist? The truth is that finding one person that possesses all these skills is very rare – and if you do find that unicorn – you probably will be outbid for their services. Instead, engaging with or developing a team that possesses these traits and skills is much more attainable. But, building a Data Science team is expensive. An alternative is to hire an external Data Science service; a team of experts that you can call on when you need to.

  • Assess your data’s suitability for Data Science model development. Once you have a Data Scientist on board or Data Science team engaged, you will need to determine whether you have enough data, the right type of data, or access to appropriate 3rd party data that can provide actionable insights. If you don’t have the data, develop a road map to acquire the data.
  • Prioritize Data Science activity. In the beginning, you can start with the low hanging fruit that will give you the biggest bang for the buck, so to speak. Then move on to more complex analyses and models.
  • Deploy and automate agile Data Science solutions. Operationalize your data science models to a production environment to produce actionable insights in an automated way. Add value to your Business Intelligence by incorporating predictive analyses into existing reports and dashboards.
  • Celebrate your success and iterate. Once you have a successful Data Science solution, you will need to update your models with new data continually to ensure the model continues to remain reliable and valid. And, don’t rest on your laurels. Continuously challenge the status quo. Make your data a key asset and strive to move towards a data-driven culture in all aspects of your business.
 
 Share on Facebook Share on Twitter Share on Reddit Share on LinkedIn
Comments Off on Data Science – Harnessing Advanced Analytics for your Business Intelligence Digital Transformation  comments 

How Businesses can Harness Intelligent CRM to Optimize Marketing and Sales Efforts

Share

Are you looking for a new way to breathe new life into marketing and sales activities? Provide your marketing and sales teams with the rich, reliable data they need to attract new prospects and engage customers. An integrated customer relationship management (CRM) solution offers an easier way to enter, access, and use the data your team needs to improve marketing efforts, identify leads, and close more sales.

The key to improving marketing and sales results is in your data, IF you can access it. Modern CRM solutions can get you closer to the data needed so you can fine-tune marketing campaigns and learn what is really driving your customers. Download this infographic, “7 Ways to Grow Sales with Intelligent CRM,” to learn how you can optimize marketing and sales with a powerful, Intelligent CRM solution.

  1. Get to know prospects: Analyze customer responses to marketing and track buying behaviors within your CRM solution to better predict which prospects are most likely to be interested in your products or services.
  2. Encourage teamwork: Coordinating information between marketing, sales, customer service, and other departments can uncover new information to improve cross-sales and up-sales.
  3. Prioritize leads: Not all leads will turn into a sale. Score leads and prioritize prospects so your sales teams are focused on the opportunities most likely produce results.
  4. Identify buying behaviors: Business intelligence features in CRM can highlight sales trends within customer buying behaviors. Learn what customers are most interested in and which customer service activities initiate a purchase.
  5. Improve marketing tactics: Determine which types of marketing activities attract the attention of prospects or customers, and result in solid leads. Fine-tune other efforts to make a greater impact.
  6. Develop forecasting accuracy: Understanding how leads turn to sales, or why and where they fail, can provide greater insight and strengthen sales forecasts.
  7. Tap into social networks: Connect to popular social networking sites to deliver even more data to your sales team, engage with customers, and monitor both your customer and competitor conversations.

When you are able to access and analyze the data your business generates each day, you can use it to optimize marketing activities and increase sales. Download the infographic and contact GNet Group for more information about using intelligent CRM to drive sales and business growth.

 
 Share on Facebook Share on Twitter Share on Reddit Share on LinkedIn
Comments Off on How Businesses can Harness Intelligent CRM to Optimize Marketing and Sales Efforts  comments