Introduction: The Necessity of Experimentation
In the digital age, companies like Amazon, Netflix, and Airbnb have paved the way for a new way of thinking: test and fail quickly. That’s where measurement and experimentation come in: techniques to quickly verify if you’re running in the right direction.
As aviation stands on the edge of change, exemplified by ventures like Flyr and Fetcherr, its leaders face a pivotal choice: to evolve or risk becoming obsolete. Experimentation, a practice embraced by tech titans such as Uber and Booking.com, can guide the way. It enables businesses to experiment in controlled settings and constantly refine strategies based on tangible results.
This cycle of continuous learning matters: Microsoft and Booking.com found that only one-third of their experiments had a positive impact on key metrics. While this seems almost disappointing, it’s a significant leap when compared to the estimated 5-10% success rate for initiatives undertaken without systematic testing.
For the airline sector, this presents a huge opportunity. From designing dynamic pricing strategies to overhauling loyalty programs grounded in real-time feedback, there are endless opportunities. The imperative isn’t merely to experiment, but to value each outcome as a stepping stone, utilizing the learnings from every test.
A culture worth aspiring to
Uber: Optimizing Ride Pricing Through Surge Algorithms
Background
One of the main challenges for ride-sharing platforms is ensuring that supply matches demand. Too few drivers during peak times can lead to missed revenue opportunities and disgruntled customers, while too many drivers without sufficient rider demand can demotivate and financially strain the driver community.
Experimentation
Uber introduced dynamic pricing, more commonly known as “surge pricing,” to address this. However, determining the right surge multiplier is complex. Uber conducted experiments by tweaking its pricing algorithms in real-time in various cities and monitoring the results.
Outcome
Through this iterative process, Uber was able to refine its surge algorithms to better predict when and where surges should be applied, ensuring a balance of driver availability and rider demand. This not only improved customer satisfaction by reducing wait times but also increased revenue during peak times.
Booking.com: Enhancing User Experience Through A/B Testing
Background
With a myriad of options for accommodations, presenting the most relevant options to users based on their preferences and browsing behavior is crucial for platforms like Booking.com. The challenge lies in understanding which site features and presentation styles resonate most with different user segments.
Experimentation
Booking.com is known for its rigorous A/B testing culture. They continuously run thousands of experiments, tweaking everything from button colors to the order of hotel listings. For instance, they tested different call-to-action phrases, image placements, and review presentation formats to understand what influences a user’s booking decision.
Outcome
These experiments have led to a more personalized and efficient user experience, resulting in higher conversion rates. They’ve found that even minor changes, such as a slight alteration in phrasing or button positioning, can have significant impacts on user engagement and bookings.
Instacart: Delivery Logistics with Advanced A/B Testing, Boosting Efficiency by 3%
Background
Instacart, a grocery delivery service, experienced rapid growth, expanding from 30 to 190 markets in just a year. Their primary challenge was efficiently dispatching shoppers to fulfill and deliver orders. The goal was to deliver groceries on time while maximizing the earnings potential for shoppers.
Experimentation
Given the challenge of conventional A/B testing in their logistics system, Instacart devised an innovative approach. Instead of splitting samples by customers or shoppers, they segmented their service areas into “zones” and then applied A/B testing by splitting samples by zone and day. They used two algorithm variants: the existing one (A) and a new one (B). Initial tests included simulations, before-and-after analysis, and difference in differences. However, these methods were not conclusive. Finally, they employed multivariate regression, which factored in various variables like zone, day of the week, and week number to assess the efficiency of the new algorithm.
Outcome
The application of multivariate regression demonstrated that the new algorithm led to a significant 3.0% improvement in delivery efficiency. By using this data-driven approach, Instacart was not only able to improve efficiency but also enhanced shopper satisfaction. This approach allowed the company to innovate at a faster pace and optimize its logistics engine in a rigorous manner.
Applications in the Airline Industry
As the aviation sector faces the challenges of a digital age, it’s becoming increasingly clear that traditional methods are challenged. To remain competitive, airlines must adopt strategies akin to those employed by tech giants like Uber, Booking.com, and Airbnb. These companies have set benchmarks in personalization, dynamic pricing, and loyalty programs through a culture rooted in experimentation.
Tech giants have found that only about one-third of their experiments actually succeed. If you aren’t learning from both hits and misses, you’re effectively wasting over 60% of your efforts. There’s no room for this kind of inefficiency in a sector as competitive as airlines.
A few examples of what’s worth measuring:
1. Hyper-Personalization
Airbnb and the like have made personalization more than just a buzzword; they’ve made it a science. Airlines can do the same. Imagine in-flight services tailored to individual passenger preferences, or special promotions based on previous trips. The catch? Experimentation. Data alone can’t create magic; it’s through iterative tests that you identify what truly resonates with your customers.
2. Dynamic Pricing
Airlines are no strangers to variable pricing, but the game has changed. Look at Uber’s real-time pricing fluctuations based on demand. How about allowing passengers to bid on fares? It’s more than just a new feature; it’s a potential revolution in how aviation approaches pricing. But you won’t know unless you try, test, and iterate.
3. Loyalty Programs
In a digital world, loyalty is currency, and stale programs won’t suffice. Experiment with offering rewards that go beyond miles. How about partnerships with e-commerce platforms or local businesses? Again, it’s about running experiments to determine what’s most engaging for your customer base.
None of these advancements can occur in a vacuum. Like tech giants, airlines must embrace a culture of experimentation. Whether you’re altering your boarding procedures to improve on-time performance or partnering with unconventional companies to expand your loyalty program, every initiative must be viewed as an experiment.
Key Challenges: Legacy Tools, Lack of New Skills, Mindset & Culture
In the airline industry, there’s an undercurrent of tension between time-tested methods and the urgent need for agile, forward-thinking practices. The industry’s ability to readily adapt to business experimentation isn’t just a question of new technology; it’s fundamentally a challenge of mindset, dated tools, and resistant culture.
Challenge 1: Skill Gaps – The Pitfalls of Simple Metrics
Let’s start with the basics. Many industry players default to simple pre/post comparisons to assess the impact of any new initiative. The problem? This method often misses the forest for the trees. These analyses overlook external variables—like seasonality or competitor strategies—that can distort results and misguide decision-making.
While the more advanced methods like the synthetic control method may seem like an upgrade, it’s not without its flaws. This approach can offer a semblance of control, yet it’s inept at picking up those marginal gains—let’s say a 1-2% uplift—that can spell the difference between red and black on a balance sheet.
Challenge 2: Legacy Tools – When ‘Tried-and-True’ Becomes ‘Outdated’
Beyond skills, there are also legacy tools that the industry holds onto, whether it’s software or contractual agreements, which often impede full-scale experimentation. These tools, built for a different era, simply can’t offer the granularity or adaptability needed for agile, data-driven decision-making.
Challenge 3: The Cultural Inertia – Hesitation Over Innovation
Then there’s culture—the not-so-small elephant in the room. The aviation industry has a storied history and an often cautious approach to change. While this caution may have its merits, it also fosters an environment that’s hesitant to wholeheartedly embrace experimentation. It’s not just about the technology; it’s about preparing minds to steer the ships differently.
A few pointers on getting started
Embarking on a journey of business experimentation in the aviation sector calls for more than just good intentions. It demands a strategic approach, designed to generate quick wins while building sustainable, long-term capabilities.
Step 1: Choose Your Battle Wisely: Scope a High-Impact Initiative
Don’t boil the ocean. Identify a single, high-impact area where the benefits of an experimental approach can be easily demonstrated. This could be your frequent flyer program or an e-commerce project—ideally, a data-rich and agile area of the business. A successful pilot in one of these domains can be a powerful catalyst for wider change.
Step 2: Assemble the Dream Team: Gather a Cross-Functional Unit
Put together a nimble team of key stakeholders: operations, data science, and finance. Make sure the group is not just diverse but also empowered with the right skill sets to implement and measure the experiment.
Step 3: Engage the Corner Office: Secure Senior Leadership Buy-In
Pick a project that already has the ear of the C-suite. When top management isn’t just signing off but is genuinely invested in the outcome, the ripple effect through the organization can be transformative.
Step 4: Seek External Guidance: Consult Industry Experts
Why reinvent the wheel? Partner with external firms or consultants who have been down this road before. A fresh perspective and best practices from sectors where experimentation is the norm can greatly accelerate your progress.
By strategically focusing on these key areas, you set the stage for a culture where experimentation isn’t a one-off initiative but a business imperative. With targeted projects, a cross-disciplinary team, C-suite buy-in, and valuable external insights, you’re not just keeping up—you’re setting the pace.
Why Embracing Experimentation is No Longer Optional for Success
In today’s ultra-competitive business landscape, the rule of the game is adapt or get left behind. Companies like Uber and Booking.com have risen to the top not just by innovating but by rigorously testing and validating each of those innovations. These firms have discovered that only about one-third of their experiments lead to positive results. It’s a huge edge knowing what 30% of your investments work and where to invest going forward.
Let’s put it bluntly: If you’re not part of the experimentation wave, you’re missing out on refining your intuition and doubling down on what’s working for your business and your customers. That’s like leaving money on the table in an industry where margins are razor-thin. Your competitors are leapfrogging, not by doing more, but by continuously learning through experimentation and getting a bit better every day.
In sectors like aviation, where the change is vital, embracing a culture of experimentation isn’t a ‘nice to have.’ It’s a lifeline. If the status quo remains unchallenged in your organization, you’re not merely stagnating—you’re effectively retreating in a race that waits for no one.
