PSA for future readers, check off Persist in Query as Jose’s screenshot that I borrowed below shows. Otherwise it goes away after you close the window.
If it is anything like the QueryOldCompanySecurity setting, don’t even reopen the Execution Settings window, or if you do, click cancel. (See linked post below.)