Proven architecture for Artificial Intelligence in collaboration with NVIDIA.
Artificial Intelligence
Strangely enough, the term Artificial Intelligence still makes me think of Science Fiction. The first thing I personally always think of is SkyNet and how it built an enormous robot army based on AI to terminate mankind from Earth. A nightmare scenario that will probably never happen. Nevertheless, forms of AI have been in use for many years and considerable steps towards advancement have been taken since 1950. From chess playing computers to service chat bots that can interact intelligently with the human users. Although this market is not standing stil and several companies are busy creating almost-human robots, there is an area where AI can do much more for mankind. Thanks to the Internet of Things (IoT), the number of sensors in the world is growing enormously. The number of people with activity trackers, medical devices and mobile devices is still increasing every day. Cars, factories, etc. are equipped with hundreds or thousands of sensors that monitor each component and report on usage and whether higher efficiency can be achieved through modification or replacement.
The amount of data generated every day is growing rapidly and data is increasingly seen as the new gold. Just like gold, data in its raw form has a certain value, but its value increases when processed appropriately. When the data from the sensors is transformed into information, the value increases enormously. After all, what could be better than a highly efficient car that can self-report to the garage when repair work is needed?
As a result, short or zero waiting time for the replacement part will be required, as the order itself has already been made as part of the process. The same logic applies, of course to human lives. How wonderful would it be if a patient’s pacemaker is monitored 24 hours and possible device failure is detected on time before the patient’s life is put at risk? Hopefully one day with the help of AI and IoT this would become a reality.
What is the biggest challenge of a huge amount of data? It’s impossible for humans alone to process it or even comprehend it. The number of data analysts required to keep up with the data flow would be enormous, to say the least. To help with this, AI comes in around the corner. Thanks to AI in combination with Machine Learning and Deep Learning, data processing can be automated and with the aid of self-learning software, this can be done more efficiently. This would enable a lot of human data scientists to put an army of virtual AI “data analysts” to work. Of course, leveraging the power of AI would be impossible without the right infrastructure, this is why the announcement from NetApp and NVIDIA is so exciting.
Announcement NetApp ONTAP AI
On August 1st 2018, NetApp announced a collaboration with NVIDIA on bringing an AI proven architecture to the market. The solution is called NetApp ONTAP AI and is built on a number of various components that have been validated for different AI workloads.
The platform, called ONTAP AI, offers an infrastructure stack based on NetApp All Flash storage and NVIDIA DGX-1 supercomputers that is very suitable for this crucial training phase of AI. Thanks to NetApp’s Data fabric approach, an AI / Deep learning pipeline from Edge to Core to the Cloud can be delivered, in which the different pieces fit efficiently
NetApp AI offers an architecture that is simple to deploy and is based on a highly scalable platform. Within the Data pipeline, various NetApp storage products can be used, which offers extra flexibility for the future. By using NetApp trident for the automated provision of NFS storage for the NVIDIA GPU Cloud Deep Learning stack, the management burden is minimized. Thanks to the collaboration between NetApp and NVIDIA, a Single Point of Contact support is provided for the entire AI architecture.
What kind of environments is this interesting for?
Embracing AI is no longer a choice of the IT department. It’s expected that business processes will increasingly demand more data insights. This will generally translate into higher demand for more storage and computing power. Unfortunately, AI cannot be treated the same as a normal workload because due to the data quantity, the workload itself could be huge. The workload could quickly escalate and cause a lot of unnecessary headache in a traditional infrastructure environment. The new ONTAP AI infrastructure from NetApp and NVIDIA, has been specifically designed and built to scale seamlessly from a very small environment to an environment with large numbers of storage systems and supercomputers. Although the validation environment is based on the NetApp A800, the NVMe flagship of NetApp, the smaller capacity AFF models like the A700, A300 and A220 are also supported.
Of course the solution has been validated by the NetApp/NVIDIA team. Although the numbers are mostly interesting for the Data scientists among us, they look pretty spectacular:
With the NetApp DataFabric and NetApp AI, the Data-pipeline can look like this:
This solution has been engineered for some time and therefore there are also a couple of reference cases already available. One of those publicly available ones has some detailed implementation information. Digital Greenhouse, part of Cambridge Consultants, developed a solution based on ONTAP AI. As one of the biggest AI research facilities worldwide, they were able to simplify and accelerate their Data Pipeline with ONTAP AI. For more details on the implementation, read the full customer success story
More information
Artificial Intelligence is a new play field for many of us. A lot of research around applying AI has been done already. If you’d like to learn more about the NetApp ONTAP AI, make sure you read:
- Announcement press release
- Announcement blog
- Solution brief SB-3939
- Reference architecture NVA-1121
- Edge to Core to Cloud white paper WP-7271
- Cambridge Consultants success story
- IDC Tech Spotlight doc
- AI solutions page with NVIDIA anchor link
- ONTAP AI product page
We, at The Storage Professionals as a trusted NetApp partner, will be happy to help you get your infrastructure AI-ready. Please feel free to contact me at richard.vandantzig@tsp.nl or visit our page www.tsp.nl for contact information.
Dive Virtual 
Leave a comment