The find() command is fantastic for retrieving documents, but what if you want to perform calculations? What if you want to find the average price of all laptops, or calculate the total sales for a specific month?

In a relational database, you would use complex SQL queries with GROUP BY and JOIN clauses. In MongoDB, you use the Aggregation Framework.

The Aggregation Framework is the most powerful feature in MongoDB. It allows you to process massive amounts of data, transform it, and return computed results in real-time.

---

What is an Aggregation Pipeline?

Think of an assembly line in a car factory. At the beginning of the line, you have a raw piece of metal. It moves to the first stage, where it gets stamped into a door. It moves to the next stage, where it gets painted. Finally, it arrives at the end of the line as a finished product.

An Aggregation Pipeline works exactly the same way.

Instead of a car, you pass your collection’s documents into the pipeline. The documents flow through multiple Stages (like $match, $group, $sort). Each stage transforms the data in some way before passing the newly transformed data to the next stage.

You write an aggregation using the aggregate() method, passing in an array of stages:

db.orders.aggregate([
  { stage1 },
  { stage2 },
  { stage3 }
])

Let’s look at the most common pipeline stages.

---

$match (Filter)

The $match stage works exactly like the find() command. It filters the documents passing through the pipeline.

Best Practice: Always put $match as early in your pipeline as possible. Filtering out unneeded documents early means the rest of the pipeline has to process significantly less data, saving massive amounts of memory and CPU.

// Find only orders that have a status of "completed"
db.orders.aggregate([
  { $match: { status: "completed" } }
])

---

$group (Calculate & Aggregate)

This is where the real magic happens. The $group stage groups documents together based on a specific “key” (like grouping all orders by the customer’s state). Once grouped, you can perform mathematical accumulators like $sum, $avg, $max, or $min.

Every $group stage must have an _id field, which defines what you are grouping by.

// Calculate the total sales revenue per state
db.orders.aggregate([
  { $match: { status: "completed" } }, // Filter first
  { 
    $group: { 
      _id: "$customerState",           // Group by the customerState field
      totalRevenue: { $sum: "$price" } // Add up all the prices in that group
    } 
  }
])

(Notice the $ in front of "$customerState" and "$price". Inside an aggregation, a string starting with a $ means “look at the value of this field from the incoming document”.)

---

$project (Shape the Output)

The $project stage works exactly like Projection in a find() query. It allows you to include, exclude, or rename fields in the document before passing it to the next stage.

You can even use it to create entirely new fields based on mathematical operations!

db.orders.aggregate([
  { 
    $project: { 
      _id: 0,                           // Exclude the _id
      productName: 1,                   // Include the productName
      finalPriceWithTax: {              // Create a new field on the fly!
        $multiply: ["$price", 1.08]     // Multiply price by 8% tax
      } 
    } 
  }
])

---

Formatting Stages ($sort, $limit, $skip)

These stages do exactly what they do in standard queries, but they act upon the aggregated data flowing through the pipeline.

// Find the top 3 highest spending customers
db.orders.aggregate([
  { $group: { _id: "$customerId", totalSpent: { $sum: "$price" } } },
  { $sort: { totalSpent: -1 } }, // Sort by highest spend first
  { $limit: 3 }                  // Only keep the top 3
])

---

$unwind (Deconstruct Arrays)

What if you have a document with an array, and you want to output a separate document for every single item in that array? You use $unwind.

Imagine an order document looks like this: { _id: 1, items: ["Laptop", "Mouse", "Keyboard"] }

db.orders.aggregate([
  { $unwind: "$items" }
])

The output would now be THREE separate documents flowing through the pipeline: { _id: 1, items: "Laptop" } { _id: 1, items: "Mouse" } { _id: 1, items: "Keyboard" }

---

$lookup (Left Outer Join)

Wait, didn’t we say MongoDB is NoSQL and doesn’t have JOINs? Well, sometimes you just really need to join data from two different collections.

$lookup performs a left outer join to an unsharded collection in the same database.

// Combine data from the 'users' collection with their respective 'orders'
db.users.aggregate([
  {
    $lookup: {
      from: "orders",           // The collection to join
      localField: "_id",        // The field from the input document (users)
      foreignField: "userId",   // The field from the joined collection (orders)
      as: "userOrders"          // The name of the new array field to output
    }
  }
])

This will output the User document, but it will now have a new array field called userOrders containing all of that user’s specific order documents!

---

$count

Simply returns the number of documents currently flowing through the pipeline.

// Count how many orders were completed in California
db.orders.aggregate([
  { $match: { status: "completed", state: "CA" } },
  { $count: "totalCaliforniaOrders" }
])

---

$facet (Multi-Faceted Pipelines)

$facet is a mind-bending stage. It allows you to process multiple, independent sub-pipelines concurrently on the exact same set of incoming documents, and then combines their results into a single document.

This is incredibly useful for building e-commerce filter sidebars (e.g., counting how many items are “Red”, while simultaneously calculating the average price, all in one query).

---

Outputting Results ($out and $merge)

Usually, the result of an aggregation is just printed to your screen or sent back to your application. But what if you want to permanently save the results of your heavy calculations directly into a new collection?

$out

The $out stage must be the absolute final stage in the pipeline. It takes all the resulting documents and writes them to a new collection. (Warning: It will overwrite the collection if it already exists!)

db.orders.aggregate([
  { $group: { _id: "$customerState", totalRevenue: { $sum: "$price" } } },
  { $out: "revenue_by_state" } // Creates a new collection with the results!
])

$merge

Like $out, this writes results to a collection, but instead of replacing the entire collection, it gracefully merges the new data with the existing data. This is perfect for materialized views or daily analytic jobs.

---

Summary

The Aggregation Framework completely separates MongoDB from simple key-value stores. By combining $match, $group, $project, and $lookup, you can write incredibly sophisticated analytic queries without ever moving the data out of the database.

Next, we are going to look closely at how we should be organizing our data to avoid needing $lookup in the first place. Get ready for Module 10: Relationships!