Prince Louis delighted the public as he pulled funny faces and chatted with his great-grandmother the Queen