Mountain Duck, based on the solid open-source foundation of Cyberduck, allows you to mount remote volumes in Finder on OS X and File Explorer on Windows.
All major protocols are supported to connect to just about any server or cloud storage, including WebDAV, Amazon S3, Microsoft Azure, OpenStack Swift, FTP, and SFTP.
Contrary to other solutions, installation on OS X requires no administrator privileges and zero system extensions.