We discovered that the Hippovend system we have been using in-world for years has suddenly, and without so much of a whisper from HippoTech, vanished into the ether.
Numerous customers began to reach out to let us know that they had paid for products that were never received. Upon investigating the situation we discovered that we were unable to access the web interface for the Hippovend system. The vendors still had the product inventories in memory, so they could browse and listen to the selected item. However, if a customer tried to purchase the item, the L$ would be transferred but since the Hippovend servers were gone, the delivery transaction would not complete…. all without any sort of error notification. Without access to the web interface, we cannot see the transaction logs to determine which products were purchased. The Linden transaction history logs only show which vending machine was used.
What Went Wrong?
Hippo Technologies brief mention of the failure was posted and detailed Rackspace crash and that the backup databases were lost, so HippoTech is effectively walking away from the service entirely. This means all HippoTech products that use the web interface are now dead in the water. The customer login link goes to their blog excuse post and their Contact page is essentially 404 as well. CasperTech also posted details about the fall of HippoTech.
This HippoVend failure only affects our in-world shop at the Skybeam sandbox. All of our products are still listed on the SL Marketplace and our sound previews here on the main website are still playable. I have placed a “temporarily closed” notice at the entrance of our in-world location to let visitors know that the vendors are not to be used for purchases. I have left the vendors online temporarily in case there are customers out there that need to get me a list of items that they have yet to receive. Once the vendors are reset, they will never work again.
What Happens Next?
We are attempting to determine what our next steps will be for our in-world shop. CasperVend is a potential solution that we are looking into but as of now we do not have an ETA for our Skybeam location to return to normal operation.
We will post updates as new information becomes available.