Fort Orange (Dutch: Fort Oranje) was the first permanent Dutch settlement in New Netherland; the present-day city and state capital Albany, New York developed near this site. It was built in 1624 as a replacement for Fort Nassau, which had been built on nearby Castle Island and served as a trading post until 1617 or 1618, when it was abandoned due to frequent flooding. Both forts were named in honor of the Dutch House of Orange-Nassau. Due to a dispute between the Director-General of New Netherland and the patroonship of Rensselaerswyck regarding jurisdiction over the fort and the surrounding community, the fort and community became an independent municipality, paving the way for the future city of Albany. After the English reconquered the region they soon abandoned Fort Orange (renamed Fort Albany) in favor of a new fort: Fort Frederick, constructed in 1676.

In 1624, a ship with 30 Protestant Walloons (people from what is today southern Belgium) landed in New Netherland; 18 of the men were sent to the location near present-day Albany. Under direction of the Dutch, they built Fort Orange roughly 2 miles (3.2 km) north of Fort Nassau, which was prone to flooding, and about five miles south of the confluence of the Mohawk River and the North River.

The Walloons were later recalled south to settle New Amsterdam. A 1628 publication on the population of New Netherland stated that "there are no families at Fort Orange ... they keep five or six and twenty (25 or 26) persons, traders, there".  In 1626, they traded over 8,000 beaver and other furs. The commander of Fort Orange and a company of men set out from the fort that year to assist the Mohican people in their war against the Mohawk, the powerful Iroquois tribe based in the Mohawk Valley to the west of the fort. The Dutch party was ambushed and three men were killed approximately a mile from the fort, roughly where Lincoln Park and Delaware Avenue are sited today. An estimated 30,000 fur pelts per year were moving through Fort Orange by 1633.

Whereas later settlement would be through the purchase of land from the Native Americans, the Dutch built Fort Orange without any consent. They continued to hold it only through the goodwill of the Mahican, and the occasional presents that they gave to the local Mahicans.^{[citation needed]}

When the Dutch established the Charter of Privileges and Exemptions in 1629 setting up the patroon system, Kiliaen van Rensselaer established his patroonship of Rensselaerswyck, surrounding Fort Orange on 24 miles (39 km) of shoreline along the Hudson River and 24 miles (39 km) inland on each side of the fort. This land patent was interpreted by van Rensselaer as including Fort Orange and the settlement that had begun outside its walls. He began purchasing and acquiring title to the lands from the Mahican.^{[citation needed]}

In 1630, Gillis Hoosett purchased in van Rensselaer's name the lands to the south and north of the fort from the natives. Later in 1630 the first permanent Dutch settlers and farmers came to Fort Orange and settled on the outskirts of the fort; their village was first called the Fuyck and later Beverwyck. In 1634 the commander of Fort Orange ordered Harmen Meyndertsz van den Bogaert west into the Mohawk Valley and Indian country, for the purpose of understanding why the fur trade had declined. The trip lasted six weeks and took Bogaert and his men through a number of Mohawk villages and into Oneida villages, at least 100 miles from the fort. This journey was recorded in van den Bogaert's daily journal which is titled, A Journey into Mohawk and Oneida Country. In the 1640s a French Jesuit priest and missionary, Isaac Jogues, described Fort Orange as "a wretched little fort ... built of stakes, with four or five pieces of cannon of Breteuil".

In 1648, dispute arose between agents of the Dutch West India Company and agents of the patroon over control of Fort Orange and the surrounding settlement. The Director-General of New Netherland, representing the West India Company, was Pieter Stuyvesant, who saw the patroon's position, power, and land as a direct threat to the West India Company's ability to profit from the beaver pelt trade in Fort Orange. Several confrontations arose over the status of the fort and the rights of settlers around it. Stuyvesant at first ordered all buildings within cannon shot of the fort to be destroyed, then lowered that circumference to that of musket shot. In response, the patroon's agent, Commander van Schlechtenhorst, decided to expand settlement to "within pistol shot of Fort Orange".

After the yearly freshets had damaged much of the fort, the West India Company decided to reconstruct the fort using stone. In response, van Schlechtenhorst declared it illegal for anyone to quarry stone within Rensselaerswyck for the fort or for anyone to sell the material to the fort's commander, Carl van Brugge. All material for the fort had to be shipped in from outside the colony. Van Schlechtenhorst claimed that Fort Orange had been illegally built on the patroon's lands, while Stuyvesant noted that Fort Orange had been built 15 years prior to the establishment of Rensselaerswyck.