https://www.channelfutures.com/wp-content/themes/channelfutures_child/assets/images/logo/footer-new-logo.png
  • Home
  • Technologies
    • Back
    • Analytics
    • Artificial Intelligence
    • Cloud
    • Data Centers
    • Desktop
    • IoT
    • Mobility
    • Networking
    • Open Source
    • RMM/PSA
    • Security
    • Virtualization
    • Voice/Connectivity
  • Strategy
    • Back
    • Best Practices
    • Business Models
    • Channel 101
    • Channel Programs
    • Channel Research
    • Digital Transformation
    • Diversity & Inclusion
    • Leadership
    • Mergers and Acquisitions
    • Sales & Marketing
    • Specialty Practices
  • MSSP Insider
    • Back
    • Business of Security
    • Cloud and Edge
    • Endpoint
    • Network
    • People and Careers
    • Training and Policies
  • MSP 501
    • Back
    • 2020 MSP 501 Rankings
    • 2020 Hot 101 Rankings
    • 2020 MSP 501 Report
  • Intelligence
    • Back
    • Our Sponsors
    • From the Industry
    • Content Resources
    • COVID-19 Partner Help
    • Galleries
    • Podcasts
    • Reports
    • Videos
    • Webinars
    • White Papers
  • EMEA
  • Awards
    • Back
    • Excellence in Digital Services
    • 2020 MSP 501
    • Top Gun 51
  • Events
    • Back
    • CP Conference & Expo
    • Channel Partners Evolution
    • Channel Evolution Europe
    • Channel Partners Event Coverage
    • Webinars
  • Channel Mentor
    • Back
    • Channel Market Intelligence
    • Channel Educational Series
Channel Futures
  • NEWSLETTER
  • Home
  • Technologies
    • Back
    • Analytics
    • Artificial Intelligence
    • Cloud
    • Data Centers
    • Desktop
    • IoT
    • Mobility
    • Networking
    • Open Source
    • RMM/PSA
    • Security
    • Virtualization
    • Voice/Connectivity
  • Strategy
    • Back
    • Best Practices
    • Business Models
    • Channel 101
    • Channel Programs
    • Channel Research
    • Digital Transformation
    • Diversity & Inclusion
    • Leadership
    • Mergers and Acquisitions
    • Sales & Marketing
    • Specialty Practices
  • MSSP Insider
    • Back
    • Business of Security
    • Cloud and Edge
    • Endpoint
    • Network
    • People and Careers
    • Training and Policies
  • MSP 501
    • Back
    • 2020 MSP 501 Rankings
    • 2020 Hot 101 Rankings
    • 2020 MSP 501 Report
  • Intelligence
    • Back
    • Our Sponsors
    • From the Industry
    • Content Resources
    • COVID-19 Partner Help
    • Galleries
    • Podcasts
    • Reports
    • Videos
    • Webinars
    • White Papers
  • EMEA
  • Awards
    • Back
    • Excellence in Digital Services
    • 2020 MSP 501
    • Top Gun 51
  • Events
    • Back
    • CP Conference & Expo
    • Channel Partners Evolution
    • Channel Evolution Europe
    • Channel Partners Event Coverage
    • Webinars
  • Channel Mentor
    • Back
    • Channel Market Intelligence
    • Channel Educational Series
    • Newsletter
  • REGISTER
  • MSPs
  • VARs / SIs
  • Digital Service Providers
  • Cloud Service Providers
  • CHANNEL PARTNERS ONLINE
 Channel Futures

Events


Ali Ghodsi cofounder and head of engineering at Databricks

The Rise Of Apache Spark In The Cloud

  • Written by Mike Vizard
  • April 6, 2015
Databricks added a job scheduling capability to make it simpler to automate workflows involving the Databricks Cloud service. Deployed on Amazon Web Services (AWS), solution providers across the channel should take note of the rise of Apache Spark.

For all the excitement over Hadoop initial interest in the big data management framework was fairly limited to batch-oriented jobs. But the emergence of Apache Spark as a framework for running big data applications that run in-memory on top of Hadoop has the potential to dramatically extend the scope and reach of where and how Hadoop gets employed as a platform.

With the price of memory falling so sharply, many organizations now want to deploy big data applications that run in memory. Of course, it takes a fair amount of infrastructure resources to run Hadoop, which accounts for why so many Hadoop deployments are moving into the cloud.

Naturally, one of the first vendors to apply that same concept to Apache Spark is Databricks, which is led by the team that developed the Apache Spark framework in the first place. Most recently, Databricks added a job scheduling capability to make it simpler to automate workflows involving the Databricks Cloud service.

Deployed on Amazon Web Services (AWS), solution providers across the channel should take note of the rise of Apache Spark. There are now more than 500 plus Apache Spark deployments in production. Databricks itself only has about two dozen customers at the moment, but as one of the few places where organizations can access Apache Spark in the cloud that number should rapidly increase in the months ahead.

Ali Ghodsi, co-founder and head of engineering at Databricks, said the reason for this is that Databricks exposes Apache Spark via a simple RESTful application programming interface (API) that enables organizations to provision Apache Spark cluster in a matter of minutes.

The implications of the rise of Apache Spark are manifold. Most significantly, Apache Spark significantly expands the type of applications that can be deployed on Hadoop into the realm of real time. With multiple forms of memory dropping in price real time applications are now within the reach of a much broader number of organizations.

Click here for Talkin’ Cloud’s Top 100 CSP list

What many of those organizations can’t afford is a proprietary big data database to run applications in real time. As such, Apache Spark represents an open source alternative to in-memory databases from SAP, Oracle and Microsoft that sits directly on top of a Hadoop framework where most big data is going to be stored. Rather than having to move data in an out of Hadoop the Apache Spark approach generates a lot less in the way of data management headaches for all concerned.

It’s still relatively early days when it comes to Apache Spark adoption, but for a lot of organizations it’s already clear that the path of least resistance to building and deploying big data applications is most likely going to be via deploying Apache Spark in the cloud.

Tags: Cloud Service Providers Digital Service Providers MSPs VARs/SIs Channel Research

Related


  • Cloud Strategy
    In 2021, Cloud Governance Becomes Imperative as Adoption Soars
    CloudSphere, Fortanix and MontyCloud all talk about what’s at stake and what partners must do now.
  • Network monitoring and management
    Malwarebytes Enhances OneView to Help MSPs' Security Business
    Security is an increasing concern for MSPs, particularly amid COVID-19.
  • Risk, Threat, Vulnerability Assessment
    Tenable Research: Publicly Known Vulnerabilities Increased in 2020
    Ransomware was by far the most popular attack vector in 2020.
  • Business hands together, diversity
    Survey: A Closer Look at Diversity in the Channel Community in 2021
    The survey is designed to gain a deeper understanding of our channel community, amplifying diversity and inclusion.

Leave a comment Cancel reply

-or-

Log in with your Channel Futures account

Alternatively, post a comment by completing the form below:

Your email address will not be published. Required fields are marked *

Related Content

  • Daunting Year Ahead: 12 Cybersecurity Predictions for 2021
  • High Stress, Demand for Security Services Among MSP Pain Points
  • SolarWinds Cyberattack Likely Affected Thousands Worldwide
  • Working with an MSP Crucial for More SMBs, Midmarket Businesses

Galleries

View all

New, Changing Partner Programs: AWS, Tech Data, Avaya, Verizon

January 11, 2021

Industry Perspectives

View all

The Rise in Remote Work Increases the Need for Patch Management

January 27, 2021

Partners Share Their 2021 Goals—and Plans for Achieving Them

January 26, 2021

The Importance of Being Security-Centric

January 22, 2021

Webinars

View all

Your Network Perimeter Has Changed

February 18, 2021

In Case of Emergency: The Importance of Proactive Critical Event Management

February 23, 2021

How Managed Hosting Providers Thrive with the Alternative Cloud

February 24, 2021

White Papers

View all

Product Brief: Kaseya VSA Integrated Workflows with BMS and IT Glue

January 26, 2021

Why Subscription Business Model

January 15, 2021

The Ultimate MSP Guide to Sales Efficiency

January 14, 2021

Upcoming Events

View all

Channel Partners Virtual

March 2, 2021 - March 4, 2021

Channel Partners Conference & Expo

November 1, 2021 - November 4, 2021

Videos and Fastchats

View all

FASTCHAT: How SOAR Eliminates Security Challenges and Elevates Service Provider Revenues

January 6, 2021

Happy Holidays from Channel Partners & Channel Futures!

December 21, 2020

FASTCHAT: How Old, Unpatched Technologies Are Creating New Security Threats for MSPs and Their Customers

December 3, 2020

Twitter

ChannelFutures

Building an efficient and profitable #patchmanagement practice in 2021 @ConnectWise #cybersecurity #endpoint #MSP… twitter.com/i/web/status/1…

January 28, 2021
ChannelFutures

International effort takes down #Emotet botnet, but @Netenrich, @digitalshadows, @Vectra_AI say it will be back.… twitter.com/i/web/status/1…

January 28, 2021
ChannelFutures

.@Microsoft @Azure’s Tyler Bryson, new U.S. channel head, talks priorities at @GetNerdio’s #NerdioCon… twitter.com/i/web/status/1…

January 27, 2021
ChannelFutures

.@CryptoStopper hires @GetChanneled to build partner program, act as virtual channel chief. #ransomware… twitter.com/i/web/status/1…

January 27, 2021
ChannelFutures

MSSPs, check for this ‘novel’ social engineering threat from North Korea. #Google. dlvr.it/RrTS9J https://t.co/2mDcnNvkHz

January 27, 2021
ChannelFutures

.@keepersecurity report shows financial sector heavily targeted by #cybercriminals. dlvr.it/RrTBPz https://t.co/joTBNeb2MT

January 27, 2021
ChannelFutures

.@Trustwave unveils new global referral partner program. #cybersecurity dlvr.it/RrT9Td https://t.co/amXCw33UsF

January 27, 2021
ChannelFutures

Ecosystem security provider Cyberpion launches first #partnerprogram. dlvr.it/RrSnxK https://t.co/g7Po3jq8iw

January 27, 2021

MSSP Insider

Newsletters and Updates

Sign up for The Channel Report, Channel Futures Update, MSP 501 Newsletter and more.

Live Channel Events

Get the latest information on the next industry-leading Channel Partners event.

Channel Partners Online

Want more? Find more channel news and analysis on our sister site, Channel Partners.

Media Kit And Advertising

Want to reach our audience? Access our media kit

DISCOVER MORE FROM INFORMA TECH

  • Channel Partners Online
  • Channel Partners Events
  • MSP 501
  • MSSP Insider
  • IoT World Today
  • Webhostingtalk

WORKING WITH US

  • Contact
  • About us
  • Advertise
  • Newsletter

FOLLOW Channel Futures ON SOCIAL

  • Privacy
  • CCPA: “Do Not Sell My Data”
  • Cookie Policy
  • Terms
Copyright © 2021 Informa PLC. Informa PLC is registered in England and Wales with company number 8860726 whose registered and Head office is 5 Howick Place, London, SW1P 1WG.
This website uses cookies, including third party ones, to allow for analysis of how people use our website in order to improve your experience and our services. By continuing to use our website, you agree to the use of such cookies. Click here for more information on our Cookie Policy and Privacy Policy.
X