Skip to main content

How Instacart remade its systems to handle a 500% jump in order volume


Shelter-in-place orders and business closures motivated by the coronavirus pandemic have prompted a rise in demand for goods delivery. A report published by CivicScience suggests that 22% of U.S. adults surveyed are having food delivered, up from 19% in early April. A separate RBC Capital Markets study found that 41% of 1,500 respondents expected to buy groceries online within the coming months. And eMarketer estimates that online U.S. food and beverage sales will grow 23.4% to over $32 billion by 2021, making it the fastest-growing category in ecommerce.

Instacart is among those benefiting from this historic growth. In spite of expenditures that will expand its workforce of service agents and shoppers by hundreds of thousands, it made a net profit of about $10 million in April, the Information reported. According to that report, Instacart customers bought close to $700 million worth of goods in the first two weeks of April, helping reverse the company’s loss of $300 million in 2019.

In an interview, Instacart CTO Mark Schaaf told VentureBeat that keeping pace with the influx of business has required technical innovations across engineering, product and design, and data science teams. With order volumes increasing 500% year-over-year, app downloads up 218% month-over-month in March, and three years’ of projected growth occurring within a period of 30 days (from March 10 to March 31), behind-the-scene changes had to be made to keep the site, apps, and service stable.

Infrastructure and AI improvements

Schaaf says that through the month of March, Instacart scaled out the infrastructure supporting its customer-facing app, shopper app, enterprise software, and advertising engine ahead of a 20% day-over-day growth trajectory. This entailed “breaking” and resetting various backend systems to align projected growth on the platform with order growth occurring as shelter-in-place mandates rolled out across North America.

VB Event

The AI Impact Tour – Atlanta

Continuing our tour, we’re headed to Atlanta for the AI Impact Tour stop on April 10th. This exclusive, invite-only event, in partnership with Microsoft, will feature discussions on how generative AI is transforming the security workforce. Space is limited, so request an invite today.
Request an invite

For instance, as Instacart expanded delivery to 2,500 new store locations with 35 new retail partners beginning in March, it made changes to its configuration datastores to better handle “dramatically” different read and write patterns corresponding with fluctuations in orders. These upgrades were often made multiple times a week.

Instacart

Above: Day-to-day traffic trends through March 2020.

Image Credit: Instacart

Instacart implemented changes aimed at narrowing the gap between what’s in-store and shown on its apps to address a nearly 30% drop in ordered items being found (at the beginning of March) and over 35% month-over-month growth in average customer basket size. Engineers at the company built a new tool that allows retailers to specify maximum item quantities, enabling Instacart to track retailer-specific item maximums across 350 partners and over 25,000 stores daily.

Instacart also made changes to its item availability model. The model predicts whether products were found by shoppers, considering among other variables the time, day of the week, and item-specific metadata. To better anticipate which products might be in short supply, the company doubled the rate at which it ran the model from every two hours to every hour while at the same time narrowing the range of historical data the model considers from 30 days to one week (and in some cases three days).

Elsewhere, Instacart doubled the number of products the item availability model scores, allowing the automatic filtering of low-stock items in search results. At stores where items like yeast, flour, paper towels, meat, or toilet paper are in relatively short supply, customers might not see them listed for purchase through Instacart’s apps. Alternatively, they might encounter “out of stock” badges next to products that are likely to become unavailable.

“We have long used the item availability model, with a few key goals. First, to remove items from the storefront that are unlikely to be found in the store based on near real-time data. And second, to set expectations regarding stock levels and create an experience where customers can select replacement preferences for items that may not be in the store,” said Schaaf. “As demand within different categories continues to ebb and flow, we’ll readjust our availability and replacement models as needed.”

Instacart also reengineered its fulfillment capacity model to meet the needs of customers marooned at home. The company claims the model that calculates how much order capacity is available at any time now runs twice as fast — every two minutes — taking into account new store hours and restocking times. For end users, this translates to faster fulfillment time refreshes on Instacart’s app and site, says Schaaf.

“I’m proud of the work our teams have done to accommodate new customers and shoppers over the last two months and meet the demand we’ve seen,” added Schaaf. “If you look at the hard data, things are beginning to even out. Last week, nearly 95% of all orders were delivered the same or next day — up from about 50% of all orders being delivered the same or next day at the end of March.”

Feature rollouts

The pandemic accelerated the rollout of features Instacart began piloting in recent months, including Leave at My Door Delivery and Mobile Checkout. The company pushed Leave at My Door Delivery, which instructs Instacart shoppers to drop items off at customer-specified locations, to all users in March. It similarly fast-tracked Mobile Checkout — which lets shoppers pay at store registers using Apple Pay or Google Pay — and expanded pickup from more than 400 new store locations.

Instacart also greenlit a streamlined alcohol delivery feature that allows shoppers to scan a customer’s ID upon delivery, eliminating the need for signatures on those customers’ phones. (Unless it’s required by the retailer or the state they deliver in, shoppers won’t get prompts to collect signatures; they can instead enter the date in the signature box.) In a period of about two weeks, the company launched Fast and Flexible ordering, which lets users skip normal delivery windows and opt for the first open window in their area. And it debuted a new prescription management tool with Costco that lets customers order prescription medications online — along with their groceries — in 42 states.

Instacart

Today, Instacart launched a new in-app safety hub intended to provide access to safety tools, resources, and guidelines for part- and full-time shoppers. In partnership with ADT in the coming weeks, a “Get emergency assistance” button will connect workers with emergency services if need be, sharing their GPS location and contact details with EMTs. Beyond this, shoppers will see new guidance on safety organizations and injury protection, as well as about topics like shipping and delivery safety.

Also today, Instacart began rolling out shopper identity verification, which prompts shoppers to take periodic photos of themselves to ensure their accounts remain secure. These arrive alongside new perks via partnerships with Sprint, CarAdvise, and Good Same that provide shoppers with savings on cell phone coverage, automobile maintenance, and roadside assistance.

“We’re seeing the highest customer demand in Instacart history and have more active shoppers on our platform today than ever before, picking and delivering groceries for consumers across North America,” said Schaaf. “COVID-19 has changed the way everyone gets their food, and it’s required us to redesign our products and systems. [The new] features … better reflect everyone’s new normal — bulk buying and low item availability.”

Responding to workers

Customer-facing improvements aside, Instacart — which delivers from over 25,000 stores in more than 5,000 cities in North America — has been criticized for its response to shoppers claiming they’re being put in harm’s way. While the company announced in early spring that it would provide paid leave to part-time workers who had been diagnosed with COVID-19 or who were placed in mandatory quarantine, obtaining those benefits requires a COVID-19 test or a letter from a public health agency. And shipments of the company’s promised safety kits containing face masks, hand sanitizer, and thermometers were reportedly delayed by several weeks.

In March, workers staged a protest demanding that the company provide more flexible paid leave and $5-per-order hazard payments.

Bowing to pressure, Instacart recently introduced a bonus program and additional promotions that let full-service shoppers earn more depending on regional demand. (The company says it has spent $20 million over the last few weeks to support shoppers.) In addition, it rolled out a Wellness Check feature to assess whether workers are experiencing COVID-19 symptoms by asking them questions via Instacart’s shopper app. If the shopper indicates that they aren’t feeling well, Wellness Check directs them to contact their health care provider and refrain from shopping until they feel better.

“Throughout our time sheltered in place, the team has done a lot of work to open up more aisles of the grocery store for delivery and pickup,” said Schaaf. “[And] our … teams are committed to the health and safety of the shopper community.”

VB Daily - get the latest in your inbox

Thanks for subscribing. Check out more VB newsletters here.

An error occured.