After stripping away 160 years of questionable interventions, architect Broderick Ely’s approach for this project was one of re-imagining – not what was, necessarily, but what might have been. Ely, director of Melbourne’s B.E Architecture, pieced together the original form from what little remained. Though everything behind the façade has been rebuilt, the floor plate of the building borrows from the original – formal, Victorian-proportioned rooms occupy the front portion of the house, with a light-filled modernist addition at the rear.

Weaving through the house’s stately rooms is a delicate narrative about the role of art in architecture. As the project developed, client and architect also worked together on assembling an enviable art collection for the house.

It begins at the front door, with elegant panels of agate leadlight in earthy browns and oranges, inspired by German artist Sigmar Polke. “The curve above the door was original so we reused that form,” Ely says. “The door is based on a French design from the 1780s – it’s not historically correct but it feels like it might have been.”

The entry hall and formal lounge beyond reveal the fine craft and idiosyncrasy of the design. Finely sculpted timber skirting boards step a foot or so up the wall. Variation in the timber marks the transition from one board to the next and timber dowelling recalls the crafts of yore.

In the lounge, curved cornices create a seamless transition from wall to ceiling. It is almost monastic. An inbuilt artwork – a whimsical coronet in white plaster with small birds perched at intervals – hangs from the ceiling: a Brâncusi-ish folly that looks as though it grew in the space.

It’s clear that Ely’s focus on art goes beyond mere acquisition, to encompass thoughtful curation and dynamic relationships between pieces. One fine example is the juxtaposition of a large-scale piece by Brook Andrew depicting an indigenous scene in gold and black against a Polly Borland portrait of Queen Elizabeth II. Together, they become an unfinished conversation on colonialism and Aboriginal history.

“I have a real dislike of subjectivity in the architecture itself,” Ely says. “In houses, you have a huge responsibility to your client, to do the right thing for them and for other people. I don’t think it’s a stage on which to wave your flag – you have to take your ego out of it.”

From the lounge, a pair of doors opens to a compact dining-room-cum-wine-cellar. With a flat-pack-style marble table at its centre, the room is walled with black inbuilt wine fridges. It could be a chic wine bar – dim, moody and the perfect solution for one of those window-less middle rooms that Victorian terraces often have.

Beyond the client’s study, the house transitions from the original volume into the new. It opens out here – it is airy and light, with banks of steel-framed windows that might have been designed by Khan or one of his contemporaries. “We imagined this space like a kitchen in a manor house, with the kitchen and the meals area,” Ely says.

An island bench of hand-sanded steel lines up with a rough-hewn table made from an elm that once stood by the Melbourne Cricket Ground. “They planted them for the 1956 Olympics, to welcome the English team,” Ely explains. “Eventually they had to take some of the elms out. We joined two panels down the middle – it feels like something you might have found in Japan.”

At night, an illuminated artwork draws guests through this space to a sunken courtyard at the rear. As the stairs descend past a spindly maple, illuminated letters spell out a quiet aphorism: Heaven is a place where nothing happens. The piece is by Glasgow artist Nathan Coley. “It’s the first piece of artwork I bought with the client so it’s pretty special,” Ely says.

Below, the courtyard leads into a series of cavernous rooms that wrap around a deep-set pool. Encased in grey stone, it is both luxurious and monolithic – a Soviet dream of a Roman bath. It leads through to a private gym, onsen and steam room, along with space for smoking, curing and storing food. These dark lower-level spaces provide an unlikely moment for art and, perhaps, this heightens the wonder of the Christian Capurro video artwork projected through a glass door.

The house’s upper level hosts the master bedroom, a guest bedroom and two children’s bedrooms. As below, the rooms towards the front were shaped by Victorian scales and finishes – a dressing room wrapped in distressed Venetian mirror leads into the master bedroom, which gazes out to the park over the road. One wall is filled by a 1950s’ Italian sideboard backed with an artwork by Imants Tillers, inspired by a quote on religion from Irish author Colum McCann. Alongside, a serene en suite – all heavy-set grey stone with sculpted edges – offers a moment for quiet repose.

At the rear of this level, a linear laundry steps out to a rooftop terrace. Its garden beds have already produced a bounty of herbs, fruit and vegetables but it is still awaiting a grapevine-clad pergola. From here, another flight of stairs leads up to a second terrace that gazes back to the city skyline a few kilometres away.

For Ely, the nuanced interplay of art and architecture is what makes this house work. “Though everything is turned up, there’s an evenness here,” he says. “The architecture, the artwork, the furniture – it’s all in balance. And hopefully the artwork makes the architecture better, and the architecture makes the artwork better.”