Symptoms:

After being logged in, you log out, only to still see the Toolbox as you browse the site.

What is happening here is the browser cached the appearance of the pages when you were logged in, and is showing you these - complete with the toolbox - even though you are now logged out. Despite appearances, this is not a security problem, as the server will catch any attempt to make changes, and require you to login again.

Solution:

To solve the problem you need to ensure the browser asks for a fresh version of the page each time you visit it:

In Internet Explorer: change this browser setting:
Tools / Internet Options / Browsing History - 'Settings' / Check for newer versions of stored pages > 'Every time I visit the webpage'