Fairhaven Park's "Rose Garden" house was constructed as a caretaker's residence in late 1914 on a "commanding position on the new portion of the park, a modern cottage erected at a cost of $3,500." The structure was built by the Larrabee associates, under the direction of Larrabee's Bellingham operations manager, Cyrus Gates.
Gates had just overseen completion of a virtually identical caretaker's home at Woodstock Farm, and "a commodious pavilion and rest room" nearby in Fairhaven Park. After Charles Larrabee's death the following year, Gates constructed a third caretaker's house in Larrabee State Park, on the scenic highway that would link a series of Gates-Larrabee legacy sites: Lairmont Manor, Fairhaven High School, Fairhaven Park, the recently adapted Fairhaven Fire Station, Arroyo Park and Larrabee Park. Efforts to establish Chuckanut Drive as a scenic road began in the late 1890s and Fairhaven Park was to anchor it firmly in the City of Fairhaven, whose founders believed parks and scenic drives were vital to modern city life.
A popular youth hostel opened in the Caretakers House during the 1990's but soon closed as it was too small to be profitable. The house remained vacant from 1998 until 2008 when the Center for Local Self Reliance obtained a license agreement to restore the premises.