Composer is a dependency manager for PHP. Drupal core uses Composer to manage core dependencies like Symfony components and Guzzle.
There are many benefits to using Composer. In short, it allows us to systematically manage a sprawling list of dependencies (and their subsidiary dependencies). It assists with locating, downloading, validating, and loading said packages, all while ensuring that exactly the right versions for each package is used. It can be a pain, but it's far better than using either nothing or developing a home-brewed solution.
Composer in Drupal core
Drupal's composer-built dependencies are not committed (via git) to Drupal's core's repository. This is why you will not find a "vendor" directory in the core repository. Instead, composer.json and composer.lock are committed to Drupal core. These files act as a manifest for building dependencies.
Drupal.org builds Drupal core's composer-defined dependencies and packages them (along with Drupal itself) into the .zip and .tar.gz archives that are available for download on Drupal.org. So, if you've downloaded Drupal as a .tar.gz or .zip file from Drupal.org, or if you've used drush to download Drupal, then the Composer dependencies for Drupal core have already been built and provided to you.
If you'd like to manage additional dependencies via Composer (beyond the dependencies already required Drupal core) then continue reading.
Using Composer to manage Drupal dependencies
There are a few scenarios in which you might use Composer on a Drupal project.
As a Drupal site architect, I'd like to manage dependencies for an entire Drupal site via Composer. E.g., install and update Drupal itself via Composer, along with any necessary non-core modules, themes, and third party libraries for the entire Drupal site.
[7.x] [8.x] Using Composer to manage Drupal site dependencies – there are ready made Drupal composer.json templates available on GitHub – drupal-project is the best known and also includes Drush and Drupal Console for command-line site management. This page also explains how versions in your composer.json are mapped
to versions and branches on drupal.org.
[7.x] Use composer_manager to automatically curate the composer.json file in your Drupal installation's root directory. Deprecated.
As a Drupal developer, I'd like to manage contributed dependencies for a custom project (module, theme, profile, etc.) via Composer.
If you are developing a custom project (module, theme, profile, etc.) that will not be contributed on Drupal.org, you can manage dependencies for a custom project using Composer.
As a Drupal contributed project maintainer, I'd like to manage contributed dependencies via Composer.
If a contributed project maintainer wishes to add a dependency on a packagist library that is not hosted on drupal.org, they can add a composer.json file to their contributed project. Most contrib developers do not need to do this as long as their drupal.org dependencies are expressed in their info.yml files.