Head up to the path that leads to the house and you'll receive a memory link as soon as you climb up the second stairs.
Go towards where the house should be, but you'll find that it hasn't been built yet within this memory. So keep going to the left towards the path to the cliff overlooking the lighthouse.
You'll receive a memory from the conversation about the house and another one when you find River and Johnny sitting on the bench.
Approach Johnny and you'll receive another memory link, then examine River to receive the last one.
The memento is a bit tricky. Seems like you can get bugged at this stage and unable to examine the memento. To fix it, click on the backpack near the log instead of using the enter key. That should let you interact with the memento.
Press the blue button, yellow button and purple button on the left side of the puzzle. Then press the blue button on the bottom side of the puzzle and move onto the next memory.