Nevermind. I just figured it out. Instead of using POIFSFileSystem, just supply the FileStream directly to WorkbookFactory.Create(...) and it seems to work just fine.
Dim xlWB As IWorkbook = WorkbookFactory.Create(New FileStream(Me.solFilePath, FileMode.Open, FileAccess.Read))
Sorry for the false alarm.
