Fish began evolving about 530 million years ago during the Cambrian explosion. It was during this time that the early chordates developed the skull and the vertebral column, leading to the first craniates and vertebrates. The first fish lineages belong to the Agnatha, or jawless fish. Early examples include Haikouichthys. During the late Cambrian, eel-like jawless fish called the conodonts, and small mostly armoured fish known as ostracoderms, first appeared. Most jawless fish are now extinct; but the extant lampreys may approximate ancient pre-jawed fish. Lampreys belong to the Cyclostomata, which includes the extant hagfish, and this group may have split early on from other agnathans.

The earliest jawed vertebrates probably developed during the late Ordovician period. They are first represented in the fossil record from the Silurian by two groups of fish: the armoured fish known as placoderms, which evolved from the ostracoderms; and the Acanthodii (or spiny sharks). The jawed fish that are still extant in modern days also appeared during the late Silurian: the Chondrichthyes (or cartilaginous fish) and the Osteichthyes (or bony fish). The bony fish evolved into two separate groups: the Actinopterygii (or ray-finned fish) and Sarcopterygii (which includes the lobe-finned fish).

During the Devonian period a great increase in fish variety occurred, especially among the ostracoderms and placoderms, and also among the lobe-finned fish and early sharks. This has led to the Devonian being known as the age of fishes. It was from the lobe-finned fish that the tetrapods evolved, the four-limbed vertebrates, represented today by amphibians, reptiles, mammals, and birds. Transitional tetrapods first appeared during the early Devonian, and by the late Devonian the first tetrapods appeared. The diversity of jawed vertebrates may indicate the evolutionary advantage of a jawed mouth; but it is unclear if the advantage of a hinged jaw is greater biting force, improved respiration, or a combination of factors.

Fish, like many other organisms, have been greatly affected by extinction events throughout natural history. The earliest ones, the Ordovician–Silurian extinction events, led to the loss of many species. The Late Devonian extinction led to the extinction of the ostracoderms and placoderms by the end of the Devonian, as well as other fish. The spiny sharks became extinct at the Permian–Triassic extinction event; the conodonts became extinct at the Triassic–Jurassic extinction event. The Cretaceous–Paleogene extinction event, and the present day Holocene extinction, have also affected fish variety and fish stocks.

Fish may have evolved from an animal similar to a coral-like sea squirt (a tunicate), whose larvae resemble early fish in important ways. The first ancestors of fish may have kept the larval form into adulthood, as some sea squirts do today, although this path cannot be proven.

Vertebrates, in other words the first fish, originated about 530 million years ago during the Cambrian explosion, which saw the rise in animal diversity.

The first ancestors of fish, or animals that were probably closely related to fish, were Haikouichthys and Myllokunmingia. These two genera all appeared around 530 Mya. Unlike the other fauna that dominated the Cambrian, these groups had the basic vertebrate body plan: a notochord, rudimentary vertebrae, and a well-defined head and tail. All of these early vertebrates lacked jaws, relying instead on filter-feeding close to the seabed.

These were followed by indisputable fossil vertebrates in the form of heavily armoured fish discovered in rocks from the Ordovician period (500–430 Mya).