There are a lot of ways to go around this.
One would be to always render the children from the Error Handler component, while also rendering the error component (I.e. Page404) when an error occurs. This error component would be overplayed on top of the actual content using CSS and you’d have a react state inside it that controls whether it should stop showing or not (I.e. Stop showing when user clicks a close button).
Another option would be to render a modal in your app and control whether it’s shown or hidden based on whether a 404 has happened or not. Thus, when a 404 happens you change a state within an effect that causes the modal to show. When the user presses a close button the state would change again and the modal would. hide.