Consider engaging your observers before the Board meeting, so that they have a thorough understanding of the content and the process for your meeting. Once you have a detailed meeting plan, discuss with the observer when exactly s/he would like to attend. (There may be places where your Board isn't comfortable having a 3rd party present, so raise this with the observer). Once you know specifically when the observer will attend, inform your Board.

Also, there may be an opportunity to share "high level outcomes" with the 3rd party during a break in the Board meeting - this will allow the 3rd party to be aware of discussions without physically being in the room.

Further, it may be worthwhile to invite the observer to your post-meeting dinner, so that s/he can meet all of the Board members in an informal setting.