Behind the cottages, at the top of the hill, alongside Vicarage Road is Grade II* listed Penn Hall, built in early Georgian style in the 17th century with early to late 18th century alterations and additions. Andor Gomme, who was a professor of English Literature and Architectural History at Keele University wrote a lengthy description of the building. He stated that from the outside the house appears to have been built in the early 18th century, but internal evidence shows it to be a 17th century house, encased within later walls. He also stated that there is a largely unaltered 17th century two storey wing to the right of the main block, which had been used as the servants' wing in the 18th century and may always have been used as such. The rainwater heads are dated 1787, which may apply to the alterations.

The memorial to Ann Bache Sedgwick in St. Bart's Church.

They had two children who both died at an early age. John was born in 1716 and died in 1721, and Ann Bache was born in 1718 and died in 1719. Their mother, Ann died in 1728 and left her estates to her husband. Ann and Raphael had planned to rebuild the house as a hospital, but nothing was done until after their deaths.

In 1729 Raphael Sedgwick married his second wife, Ann Bradley, who died in 1747. Raphael died a few months later, also in 1747 and his estates, including Penn Hall passed on to Thomas Bradney, who had been married to Ann's late sister, Mary Bradley.

In the 1780s, Penn Hall was greatly altered by Thomas Bradney, using the Sedgwick's design, produced by the architect William Baker. Thomas Bradney was born in July 1710 and became a Justice of the Peace for Staffordshire; and High Sheriff of Staffordshire in 1752.

Penn Hall. From Stebbing Shaw's History and Antiquities of Staffordshire.

Ellen Pershouse died in 1829 and her son, William Bradney Pershouse inherited the estate. He was absent from Penn for long periods. After his death in 1843, the Pershouse family let the Hall to William Underhill, a Wolverhampton-based iron stockholder.

By the late 1840s, John William Sparrow and his family were also living at Penn Hall. The 1851 census contains the following entry:

The Sparrow family became extremely wealthy thanks to the profits from their coal mines, ironstone mines, and ironworks. William Hanbury Sparrow and his brother John Sly Sparrow purchased Bilston Mill Ironworks, which they ran with John Walker, an ironmaster. The two brothers founded W. & J.S. Sparrow & Company, and leased 120 acres of land in Stow Heath, on the western side of Stow Heath Lane where they mined coal and ironstone. In 1824 they opened Stow Heath Ironworks on part of the site, and began producing iron. Before long there were five blast furnaces, using cold blast, producing hard forge iron.

John Sly Sparrow prematurely died in 1825. Stow Heath Ironworks were a great success. In 1839, 12,756 tons of hard forge iron were produced on the site, using four of the furnaces. By 1873 there were only three furnaces, and an average of one and a quarter of them were in blast.

In the mid 1840s John William Sparrow acquired a piece of land alongside Birches Barn Road in Penn Fields, on which to build a family home. The house became known as Beckminster House, and John moved there in the 1850s.

The Underhill family remained as tenants at the hall until the end of the century. By the late 1890s the hall was owned by George Harry Bradney Pershouse who inherited it after the death of John Pershouse. George lived at Stafford House, Victoria Park, Manchester and had little interest in the Midlands or his property and estates here. He gave his tenant, Mrs. Underhill notice to leave Penn Hall when the lease to her late husband expired on Michaelmas day, September 1898. She had to leave by Michaelmas, 1900. In November 1899, he put the house and the local estates up for sale.       

There were also 122 acres of land at Lower Penn, 14 acres at Wombourne and 9 acres near Dimmingsdale. The estates covered over 388 acres in total.

In 1902 Penn Hall was acquired by Thomas Francis Waterhouse who was born at Sedgley in 1866. He became a solicitor in 1887 and had a successful practice in Wolverhampton and served as Magistrates Clerk at Wolverhampton and Sedgley.

In 1899 he joined the 3rd Volunteer Battalion of the South Staffordshire Regiment as a Second Lieutenant with 'G' Company at Bilston. He was promoted to the rank of Captain and became a Major in 1906. Two years later he was promoted to Commanding Officer of the 6th South Staffordshire Regiment. At the outbreak of the First World War he was given a Regular Army Commission with the rank of Lieutenant Colonel and coordinated the recruiting efforts in the Wolverhampton area.

In October 1914 he took command of the 1/6th Battalion and in the following year was severely wounded in France at the Battle of Vermelles when a shell exploded close by. He lost an eye and needed extensive surgery. In spite of his wounds, he returned to France in 1918 and was awarded the Distinguished Service Order. Two years later he became commander of the Defence Force troops in Wolverhampton.

Although he had an exemplary military career, his home life was very different. When he purchased Penn Hall in 1902 he engaged the architect, H. T. Hare to remodel and improve the building. The work included extensive alterations which included moving the main entrance to the south side of the building instead of the west.

Penn Hall.

By 1907 he was living beyond his means and later began to misappropriate the funds belonging to several of his clients. In 1922 bankruptcy proceedings were instigated against Thomas Waterhouse, who in 1923 appeared at Stafford Assizes on trial for misappropriation of his clients' funds. He was found guilty of stealing funds to the value of £12,684 and was sentenced to imprisonment for nine years. After his disgrace, he never returned to this part of the country. He died on the 10th September, 1930 at the age of 65, in Latchmere House Ministry of Defence Hospital at Ham in Surrey.

In 1924 Penn Hall was sold to Francis J. Gibbons, a local industrialist who had a large factory in Wolverhampton, where St. John's Retail Park is today. His firm made locks, all kinds of builders' ironmongery including doorplates, knobs, handles, geared fanlight openers, panic doors, door springs and also 'Cunard' cycles. Francis had previously been living at The Beeches on Penn Road. He stayed at Penn Hall until his death in 1948.

Wolverhampton Borough's Watch Committee (the body responsible for the police) bought the property in 1948 and converted it into a residence for 25 single policemen with garages for the police vehicles in the traffic section. Most of the internal features including tapestries, paintings and fireplaces were preserved.

In 1974 when the West Midlands Police Authority was formed, the house was sold to Wolverhampton Council for use as a school to support children and young people with special educational needs or disabilities, from birth to 25 years of age, and their families.