I’ve recently seen lots of neat Twitter posts about ChatGPT. One thing that was wicked cool and caught my eye was how the infosec community was using ChatGPT (including testing both technical and ethical limits). People were using ChatGPT to explain what blocks of code did, among other things. This really piqued my curiosity.
I started playing with ChatGPT in a generic way, asking it trivia, or to spit out a block of poweshell for something mundane like listing files and their sizes in Gigabytes and export to a .csv. Eventually, I got to thinking “What can this to do with Veeam?”
The first thing I was wanted to do was see how much it knew, but beyond simply asking if it knew what Veeam was... I wanted to test its ability to engage on a much deeper and technical level within the Veeam ecosystem.
The first place I wanted to start was VMCE. So, I used one of the practice exams questions from the Veeam ProParter test exams. Could ChatGPT return something coherent beyond a simple answer that shows it ‘understands’ the question? Before we go any further, I want to stress that I do not recommend trying to study for the VMCE, or any other exam, using ChatGPT. That should be obvious, but I’d rather say it plainly than assume. For that reason, I’ll try not to express my own opinion on what may or may not be correct with any of these responses from ChatGPT ;)
Within moments, not only did it spit out an answer.
We can see that the option ChatGPT proposes does is indeed close to one of the available options:
This was impressive but some of the steps it’s given us aren’t exact... but if you had to follow along, you’d probably be fine.
So, this is one example of a prep exam. If you’ve ever studied for a VMCE or a VMCA you’ve no doubt come across Rasmus Haslund’s (@haslund on Twitter) excellent collection of exam prep material. (And if you’ve ever passed a VMCE or a VMCA you likely have him to thank.) Being so familiar with a lot of these test exams Rasmus has put together, I wanted to try to throw some of those at ChatGPT as well. Would ChatGPT be able to digest these more real-world examples? So, I passed it one of his questions;
We can see that ChatGPT has a harder time on this one, but it’s found a troubleshooting path that also includes some of the available answers, but it’s not so sure of itself and won't commit itself to a single answer:
Still very interesting stuff, but I wanted to push the limits some more. What about a VMCA question, which are much more scenario based?
I gave it this question, from Rasmus’s VMCA study guide video series.
I decided to re-write it a bit, to see if we could get a better answer this time around;
This one is interesting because it is struggling somewhat, but seems to be on the right path, until we get to 5... Backup Acceleration is not a Veeam feature. ChatGPT may be confusing this with WAN Acceleration at best, or an entirely different backup product at worst!
So, at this point I think it may be safe to say ChatGPT won’t be replacing the know-how of VMCE and VMCA certified IT Pros just yet :p
I then started to wonder about more specific use-cases. What if I gave ChatGPT actual Veeam log snippets, or HTML report error messages?
The answer ChatGPT returned with is generic, but not entirely useless:
I also wanted to pass log errors and see how it would handle those;
Again, we get a generic, but ultimately helpful response back that would certainly guide someone in the right direction:
What about specific Veeam error messages?
I wanted to try to throw more at it, so I decided to find something even more specific. As I mentioned earlier one of the first things I noticed people doing was utilizing ChatGPT to explain blocks of code in a human readable way. Could it tell me what a Veeam specific regkey does?
Some service providers out there may be familiar with this regkey:
I was surprised by this one. It does do a good job of telling us what this regkey does, though it doesn’t go into perfect detail, leaving some information out. https://www.veeam.com/kb2282 But if you had no idea what this key did and had to ask ChatGPT, you’d know what it does.
When I told a friend and a colleague about what I was playing with, they both had the same exact reply (which was even funnier because both of their backgrounds are in Security); “I want to see it write documentation”. This would be cool. Even if it doesn’t get everything right (as we’ve seen above), it could create a good starting point for someone to take and then iron out while following the documentation. So...
As we can see, the steps are rough. Again, tab names are misrepresented, but the general message is there. Sadly, it completely whiffs on 6. (Not to mention the copy intervals for Perodic copy is on the first part of the Wizard) which is funny because on step 7. it kind of nails the retention aspect of this but it presents it as more of a footnote. But could an IT Pro take this with no familiarity of Veeam, follow along, and correct the documentation appropriately? I think so. https://helpcenter.veeam.com/docs/backup/vsphere/backup_copy_name.html?ver=110
So, while ChatGPT is very useful and can do some impressive things like spitting out entire blocks of code with just a simple explanation of what you want made, it still clearly has a long way to go. This highlights just how much is expected of IT professionals and backup administrators specifically when tasked with support something like Veeam. Even a powerful AI doesn’t have all the answers.
It does seem that ChatGPT or tools like it could be hugely useful for getting you on the right track, potentially making it a threat to services like Google Search. Ultimately, in my opinion at least, ChatGPT or tools like it will simply become another tool in the IT Pro’s toolbelt. There may come a day where part of the triage process is running errors through an AI like ChatGPT. Will vendors invest in creating their own “AI” that customers can leverage as a form of support? There is a lot of potential for this kind of tech, and it raises a lot of interesting questions!
At the end of the day, it will still be the IT Pro’s ability to thoroughly research, vet and test potential solutions. It also speaks to the need of having experienced IT Pro’s both in house and in your choice of Veeam Cloud Service Provider or Manages Service Provider.
