DevOps Q&A: Crossplane, AI Agents, GitOps Terraform, and Career Tips Transcript, AI Summary & Key Points

00:00 Heat. [music] [music] [music] Heat. >> [music] >> Heat. Heat. [music] [music] >> [music] >> Hey hey hey >> [music] >> Heat. Heat. [music] [music] >> [music] >> Heat. Heat. [music] >> [music] [music] >> Heat. Heat. [music] >> [music] [music] [music] [music] >> Heat. Heat. Heat. [music] [music] Heat. [music] Heat. [music] [music] [music] Heat. >> [music] [music] >> Heat.

03:29 Heat. [music] >> [music] [music] >> Heat. Heat. [music] Heat. Heat. [music] [music] [music] Heat. [music] Heat. [music] >> [music] [music] [music] >> Hey, hey, hey. Yet another ask us anything. Scott is back. >> Oh, hello. >> I'm back. I was also out for quite a few weeks except the last. So, we are back. We answer your questions if you have them. If you don't, then Scott is going to sing for us or tell us a joke.

05:30 >> Tell us a joke. Not that either. What you're working on something. >> What are you working on these days, Victor? >> I'm uh I'm hating Windows these days. That's what I'm doing. >> Okay. I'm hating Mac these days. Well, but that's not nothing new. >> Yeah, but it's it's it's Linux essentially. So, uh you cannot hate it. >> No. >> You just you just don't like its graphical user interface.

05:53 I understand that. >> And company. >> And you're disappointed that that PC cannot make uh pro CPUs as good as as Apple can. That's that's your problem. I know >> it's hard. Yeah. Yeah. That's the issue. >> Yeah. [laughter] You you need to carry when you travel, you need to carry whole backup for a battery so that you have energy, right? Yeah. I know.

06:22 >> I know. >> I know. >> Okay. >> Anyway, hello Sham MG Desire. Uh happy Victor Scott. Happy to see you guys again, guys. Yeah, ask your questions. Otherwise, you see how uncomfortable Scott feels when I ask him to do something. Don't do it. Don't do it to him. >> No, no. I've been working a lot actually on a bunch of lane graph agents these days, which is fun.

06:47 >> Very cool. Very cool. Scott was showing me his fancy new UI for agents. Very cool. >> Exactly. >> It's not only UI. >> He's giving up on backstage. Shame on you. giving up on backstage. That will never happen. >> Did you work on backstage yesterday? >> Yes. >> Oh, really? Okay. >> I actually worked on it today as well. >> Oh, okay. Okay. I was wrong.

07:14 I was wrong. >> Exactly. >> No, it's very interesting. Look at the data for that Spotify have released. The usage of backstage amongst developers grows as they use more AI in development. >> No, no, backstage is great. My my biggest beef I mean I had two beefs with it. First that it was ridiculous how to run it. That's now better. I give them that. >> My second beef is that it hurts my eyes to look at it.

07:48 Like the UI you showed me that looks okay. That looks nice. You look at backstage and you say, "This hurts me. This hurts my eyes. I cannot do it." >> Well, the buoy made it a lot better. Backstage UI. The new UI framework in Backstage made it a lot nicer actually. But yes. >> Okay, there we go. Uh, the first one. Scott, as understand you suggested to pack and deliver crossplman compositions with Helm.

08:14 Victor said you'll never suggest Helm for anything. Who is right? Okay. So, it is not that I recommend to deliver crossbane compositions necessarily with Helm. Um, my general uh stance is that I do not understand, like, or ever recommend using configurations uh crossplane configurations. >> Are are you are you talking now about delivering it outside your organization or internally?

08:46 No, even internally, like internally I just use typically use customize and just apply it. Exactly. Whatever. >> Um I see zero value and only negatives in using crossbin configurations. I wish that were not the case, but until I can actually bundle together additional resources that go with my composition, until I can include within their provider configs together with my provider, until I can actually add all of the elements that I need.

09:18 I don't see a reason to build an abstraction above something that requires me to still wrap that abstraction within another abstraction to provide something else. I give you that >> bundle everything into a configuration. Great. Um, but because I can't, I am against configurations. Um, >> let let me let me explain it different ways and you tell me whether I get it.

09:42 You dislike and if that's the case, I agree that actually it's 100% crossplaycentric and cannot really >> and only partialentric >> and even also only partial crossplay. It's like, okay, let's say that I couldn't include a Chyo policy with that. Fine. I don't like it, but fine. But I have an issue that the configuration can't even include all of the crossplane resources that I need to make my composition work because I need a provider config.

10:13 I need a deployment runtime config. I need an MRP. I can't include those. I I give you I give you that one. Still, if I would correct what is on the screen, I hope you're not recommending it to do it internally. Help. >> No, I do not recommend Helm. I would I will recom I would say that if it's between Helm and a configuration, I would do Helm with zero templating just as a packaging mechanism.

10:47 Meaning if you want to package it, throw it in a helm chart that has zero values. Templates that aren't templated. What do I care at that point? I do not believe that you should be templating compositions in Helm. I have no problem with Helm as a distribution engine even to be used internally as long as you have zero values because then you don't change your files.

11:11 There's no go templating anything. You just added a chart.yammo YAML file and a folder called templates with your files within it. Like cool. Like I'm fine with that. >> Yeah. No templates. Okay. You scared me for a second. I I was I I I had a mini heart attack thinking that you're you're kind of changing the tune to to help templates. >> Oh, that scared me.

11:33 Okay. Uh do you use tech insights in backstage? That's for you 100%. >> Um so the so sometimes I've used tech insights. Um, I have I have a lovehate relationship with Tech Insights. Um, because building fact retrievers and all that to actually bring things into Tech Insights is not as good of an user experience as I wish it was. Um, I love the idea of Tech Insights.

12:02 Um, I think that there are better approaches uh to doing scorecards and things like that within um, Backstage. Um unfortunately some of them things like soundcheck and whatever are behind a payw wall. Um and for example harness uh built their own uh mechanism um that you can use if you're using their you know backstage based uh IDP. Um but if we're talking purely open source yeah tech insights is great.

12:35 Um, it's just not the best experience to add additional retrievers, I would say. [snorts] >> Okay, this is going to be fun. I found myself struggling between building tools now and why even do that? If an AI agent can do that complex task, should we just pivot now into helping LLM? And I will >> no what? >> No, we should not be pivoting away from building tools.

13:06 LLNs should be building tools for things that are deterministic. The question is, is it deterministic? Is it something that you can write as a script and if you can write it as a script, have the LLM write the script because why waste the money on every single iteration that that thing is going to run? If it's a one-off, LLM 100%. If it's not a one-off and it's something that can be scripted, have the LLM write the script and then run the script.

13:32 [snorts] >> And kind of and I feel that the question is contradictory because should we just pivot now into helping LLM? What you just described is actually helping LLM, right? LLMs call tools rather I mean they don't have to. I mean they have to call some tools. They can be low level, high level, but it's similar to discussion we had before uh before the stream about graphana, right?

14:00 Kind of like that discussion would be, hey, if I would have this tool, if if my agent would have this tool, this would be so much more efficient, right? >> Exactly. It's about efficiency. When an LLM is not needed, you should not be using an LLM. Um my biggest pain point, and I hear this all the time in the consultancy world, right? Like I hear this all the time from like customers of when you talk about what their use cases are for something with an LLM and like the use case I say I'm like but like you can do that in

14:29 a 20line bash script like why would you put an LLM on it and the time it's going to take is longer. It's going to cost you money every time that you run it. Why in the world and it's going to be non-deterministic. Why in the world would you do that with an LLM? LLMs are great for certain things. That amount of things is growing as the capabilities of the frontier models goes up, but where it doesn't make sense, it's I view this very similar to Kubernetes.

15:00 If you remember back in the day, everyone like you ask people why are you moving to Kubernetes? Kubernetes, Kubernetes is the answer to everything. Kubernetes is the runtime for everything. people were taking web sphere and running it in a container in Kubernetes to say that I'm moving to Kubernetes. I have customers that ran databases in a single pod with physical NVMe drivers going straight up into that single pod on a single node.

15:28 So one pod the size of one node which was a bare metal server. What value did you get from that? Zero. But why? Because everyone thought Kubernetes was the answer to everything. And the answer is no, it isn't. Now today we understand Kubernetes is right for certain things. Lambda is right for certain things. Virtual machines are right for certain things.

15:48 Bare metal servers are right for certain things. And everything has its right place. And we figured that out with Kubernetes. People have not figured that out yet with AI. They think AI is the solution for everything. It is not. And we have to use it where it makes sense and not just try because it's the new buzzword to throw LLMs into everything to replace everything that we do.

16:11 You know, one way to look at it is that I believe that LLMs should not be replacing work that was previously done without LLMs. It should replace part of the work you were doing. And hopefully you were not doing the work that you should have scripted right the basically it's the non-re repeatable work now if you use you know very often in those conversations often I come to conclusion that actually the things that you're asking LLM to do are the thing things you shouldn't have been doing in the first place without LLM

16:58 before LLM emerged kind of like you were not supposed to do that. You were supposed to have a script or or application or something to do that for you. >> Exactly. >> Okay. Uh from Shamil exactly no templates. Helm is just an easy and familiar way to bed Python code as inline in the composition manifests and to provide simple manifest to cross crossplain render command.

17:25 ah with the because you can do file read so you can include the Python files separately and then like what you do Victor where you basically write the KCL code or whatever separate and then you embed it in you can do that easily using Helm where you basically just indent file read whatever to read that Python file to package it that's perfectly fine as long as we're not going down the go templating approach of you know whatever if this then this Exactly.

17:58 It can be go like that example of inline is probably still go templating but it's two lines. >> It is but it's a one line in a composition. It's fine. You're not adding logic in. You're just saying take this file and put it here for indents in. Fine. Like I don't care. You're doing the same thing with your TCL code that takes the Python files and throws them in with function Pythonics.

18:20 Who cares? Like that I'm perfectly fine with. Um, I just have issues when people start templating out XRDS and compositions inside of Helm charts. I'm like, what are you doing? >> Oh, uh, while waiting for the next question, I had the interesting conversation. I will not name who and when. Uh, that uh, hey, we should we want to get off that we using crossplane.

18:45 That's cool. We we should get off Helm kind of. That's cool. I support you fully internally we're talking about now, right? Yeah. and kind of when we have those compositions and we really don't like Helm and we are we we we decided to use go templating >> that's the same thing it's exactly the same thing >> except because at least it's server at least it's server side with continuous reconciliation and it has a defined API in front of it so you are getting listen I can understand using function go templating is that

19:20 initial move if If you're used to writing Helm charts, write go templating function makes an easy transition when it's already a hard transition moving from client side to server side with continuous reconciliation. all that fine as an initial step that's fine but down the road you should be moving to function pythonic for example which is my favorite or to the new crossplane devx stuff that you know was donated upstream from upbound of actually just writing like python or typescript or go code as like actual functions

19:53 um >> okay >> or use a dead language like KCL >> still hurts me still hurts me that is dead. I will I will I will not for forgive forget it soon anytime soon. I have five years of experience as a deos engineer never got the chance to work on kubernetes my corporate journey how it's going to impact my skill as a deos engineer I just know the fundamentals suggest I mean I I I do believe that excluding some special cases and so exclude special cases where there is a good reason any decent company is using kubernetes in

20:35 some capacity or another kind of whether that's 1% of the workloads or 100% I don't know But I think that companies who are not even using Kubernetes in any form are are not shouldn't exist these days excluding special cases just to be clear kind of if somebody's doing uh uh half of a millisecond type of trading and okay but they know what they're doing probably right >> I exact I listen I think that the answer to how is it going to impact your skill as a DevOps engineer.

21:14 Again, without getting to the whole question of what an actual DevOps engineer or what that even means, >> let's skip it today. >> Exactly. I would say again, >> if you are capable of learning things and you come with experience in other areas, for example, it's saying you have a lot of experience in Terraform. >> Great. So, okay. So you built Terraform in the past for Lambda functions and ECS and now you're at a company that needs that for Kubernetes.

21:46 Well, you can reuse those same skills from here to here. If you understand Linux and how processes work and how you how kernel name spaces work and how croups works and all that, the move to understanding Kubernetes is not that high. And meaning as long as you understand the fundamentals of things, then I don't think it changes your skill. So I would agree on that from the perspective of would you be able if you if you understand fundamentals to relatively quickly on board to Kubernetes the answer is yes I give you

22:22 that but and excluding the case that you might be faking it it's not a good thing on in an interview with your potential future employer right whatever they consider DevOps they're going to kubern is likely going to be a big part of it, right? So maybe fake it. Fake it. Say, >> well, I do. >> Don't fake it. spin up an EKS cluster on your own in, you know, whatever in an AWS account or GKE or whatever cloud it is or on if you have a home lab or whatever it is.

22:58 Spin up a Kubernetes cluster, deploy a few apps using Argo CD, learn the tool yourself, and then don't put Kubernetes on your CV because you're not going to lie here and put Kubernetes is one of the things you're an expert in. Go in strong with what you know well, but then when they come and ask you, okay, and do you know Kubernetes? to say, you know, unfortunately, my previous employer didn't use Kubernetes much, but actually I've done a bunch of work, you know, in AWS, whatever, where I've spun stuff up and I've

23:26 played with a bunch of different tools and I've actually learned, you know, the ecosystem. So, I haven't had the fortune of doing this by my previous employer, but I took all the knowledge that I did there and actually built this out myself because I was actually really lacking um that in the previous company, which is one of the reasons that I'm looking uh you know to move to a company like yourself, right?

23:49 And then you come off honest showing that you do have the knowledge because you did learn and like I think that that's always the best way uh to go about things. Okay, this is more a clarification. It helm can also be used to deliver compositions and everything what it needs as an OCI artifact with versioning and all this stuff. Correct? >> Of course.

24:11 >> Uh okay. Have you used spiffpire? You have a product if we have a product to run some kubernetes in cloud or on premise. Should this be part of the road map to support? I guess that depends on how low level you want to go. Spiffy and Spire are not kind of out of the box kind of solution for something. They're more like building blocks. >> Exactly.

24:34 Victor uses you would use >> Victor uses Spiffy and Spire almost every single day because that's used under the hood in Psyllium. Uh, however, I'm doubting that you've ever actually used Spiffy Inspire beyond the demo for anything like real >> because almost no one does. Um, they are building blocks that are great for a product that needs that workload identity.

25:04 However, use something like ISTSTEO to give a workload identity. Use something like Psyllium to give a workload identity. You don't need typically to deal with that low level of a concept. It really depends what you mean by what your product is that runs on Kubernetes. Is it a CNI? Then yeah, use Spiffy Inspire. Is it a service mess? Use Spiffy Inspire.

25:23 If it's a WordPress type product, uh, no, don't use Spiffy Inspire. >> Don't use WordPress. That's the correct answer. >> You I said like a WordPress, you know, like something else with >> Okay, there you go. >> Don't use Piffy Inspire. [laughter] Right. >> Yeah. Yeah, it's a building block. It's like, you know, like many people will not really ever dig and and use directly, I don't know, containerd, >> right?

25:50 >> Exactly. >> It's not it's not meant to be used by by you unless you're building something that needs that capability, right? Your own pro. >> Exactly. >> Okay. Uh how about letting the AI agents do the builds? For example, I have CLA to generate terapform templates which I fed to Atlantis to run. Should they instead emp empower an agent to do this?

26:12 By generating templates, if you mean writing terraform templates. Yes. Yes. You should be building a harness around this that basically um for example would do like hey these are the you know give it triv and you know whatever cubescape if it's kubernetes if it's terraform something like trivy with you know whatever benchmarks of security tf security checks and all of that stuff plus tf lintth tf validate whatever and basically have it do continuous loops of write validate write validate test right it pass all then go

26:48 and actually do an apply and run Terraform tests and do all of this and get it to a working state and then the last step of the agent push it to the git repo that Atlantis Terraform cloud and zero whatever tool you're using would actually go then and do the deploy itself never should agent do Terraform apply to production >> correct correct I mean essentially what you just described is all the the things that you would need to do as yourself, not as tools.

27:19 You do it with agent. In other words, >> exactly once you run in a supervisor loop to write the >> afterwards it goes into a pipeline or what whichever tool you're using. You're probably not going to do updates to it either. I'm talking about version updates yourself nor through an agent, right? That's always going to be part of your workflow one way or another, whichever tools you're using.

27:45 >> Okay. uh my so this is answer to make desired my previous company didn't also use kubernetes I passed the cka uh then I got job offer and moved to a company using kubernetes yeah >> exactly use your experience as your way in the door in the overall ecosystem learn kubernetes don't just and this is I'm sorry I will just say one thing on this I passed the ck then I got a job offer please people do not just do the CKA to have a CKA.

28:21 Um, and I don't care if it's CKA, AWS, whatever certifications, don't just get the certification to get the certification. If you're gonna learn anyways to do the CKA experience, >> learn to learn Kubernetes and then get the CKA because it will help you get in the door to show when you say, "Listen, I don't have experience in production environments, but I have learned on my own free time alongside my other job and I did the CKA."

28:49 Great. But please don't just, you know, study dumps and do the CKA uh for your own benefit. It's uh because that's visible within a day in a company and people notice that so study to study. >> Okay. Uh yes we use ECS ECR lambda. Sorry for that. For EC ECR sorry [laughter] >> for ECS and ECR I'm sorry lambda is >> yeah yeah kind of lambda is fine fineish the other my apologies.

29:23 ECR is the worst registry that exists on planet Earth and ECS is the worst container runtime that exists on planet Earth. >> But you know if you're AWS shop I I would I would still consider EC ECR kind of because it's there >> exactly it's still horrible. >> Okay. Can I get a job in DevOps with no experience? I have built full stack projects. So, so I am I I I develop things, but whenever I search on Reddit for advice, they say it's not possible to get a job in it.

29:56 It's definitely possible. It shouldn't be. I mean, in my head, DevOps is when you progress further in your journey, not when you people should not be starting. That should never be a starting role. uh uh but companies may interpret DevOps differently. They interpret DevOps as anything that is not pure development or testing or project management is DevOps.

30:30 um if companies interpret it like that >> like bare metal management and things like that which they call it operations right like you have the right like exactly it's >> there are a few other >> right >> the point I'm trying to make is that I feel that devops is apart from not shouldn't exist as a role if it exists it should be you graduate after few other roles whatever those ro roles are and they could be different.

30:58 You could be you could start as CIS admin for example which is not a sexy word anymore but kind of okay. Um but the reality is that companies employ DevOps for all sorts of reasons and basically anybody who is not one of the other roles is DevOps then of you can get a job. Yeah. Uh okay. Uh I'm Nick from West Africa. Hey Nick Cameron. Cameroon. uh transitioning as an English major because it was boring.

31:29 So I thought of DevOps just some weeks ago. What's your take on that? Any advice? Uh career move. It makes sense for you to be called DevOps. Uh especially if you're starting. I just don't think it's I don't I don't think it's right. >> Uh I think that companies don't >> think DevOps is the right place to start for anyone. Not not just because I think that it's wrong for companies to put someone in that position because I think that it requires a lot that like you need the experience.

32:01 I don't think it's good for people's own careers to do that. Um because you can easily tell that people that start in DevOps and don't have that core understanding of operations and of actual system administration and whatever um and actually come from that world hit a ceiling at some point that they just can't cross because if you don't understand the fundamentals really well you will never be a really good DevOps engineer.

32:32 You can be fine. You can be okay. But if you really want to, you know, go about and become a really great engineer, you have to start from the bottom. You have to build it layer upon layer upon layer. And yeah, it's a few years of annoying work of being a Linux CIS admin, but do it. You will learn so much from that. And then when you move to a DevOps engineer job, you will be so much better.

33:06 >> So much better. >> Um, and I will say in the past I said you could start from the dev side or the op side. Today I say start from the op side because the dev side AI is taking over. Whether that's a good thing or not, AI in development is, you know, becoming so prevalent. it still is not nearly as prevalent in the ops world. So being a CIS admin, you will it will be, but it's still going to take time.

33:32 Which is why in the IT CIS admin world, you still get to really learn hands-on. You're still doing these things. You're still doing more with your hands and which is great, I think, for learning. Um Okay. Um, do you think SAS is dead, especially for indie hackers, solopreneurs building small SAS products? I I think that the whole SAS is dead is BS. Uh, I do think that many SAS companies are in big trouble because the value proposition changes and the pricing is going to change and many other things.

34:15 But AI is not going let's say okay let from top of my head SAS products GitHub is a SAS product right that's not going away Gmail is a SAS product right Google maps is SAS product uh Uber is SAS product right they're all SAS >> anthropic is a SAS product >> anthropic is a SAS product there we go there we go right [laughter] >> okay like done I just like yeah um the answer.

34:46 SAS is not dead. I think that yes, indie hackers, soloreneurs, whatever, small SAS products, I think that if it's really small, if it's not innovative enough, it's dead. Because if it's a small thing, one of the challenges that like small things today, what used to Yeah. You could have gotten $2 a user, $5 a user on something because it would have taken us, you know, whatever 6 months a year to build something like this of full-time engineers and whatever.

35:17 If it's something small, if it's not something actually like really proprietary, innovative, something that like, you know, is extremely complex. A lot of companies are saying, "Hold on, why would I pay someone? I can spend a day and a half with an agent and build that myself and not need to pay anything and actually have it exactly the way that I want.

35:41 So I think small companies, small startups are going to have this issue majorly, not just SAS. I think that the big SAS companies are here to stay. They will stay and all of that. I think the small things are and small companies in general um small products are where the actual pain is going to hit. I also feel that it depends greatly on whether we're talking SAS for end users or SAS for companies.

36:11 If you're talking about SAS for companies, I think that companies are starting to re-evaluate economics of SAS solutions uh and do it seriously, right? That does not that that does not mean that tot type of evaluation. But uh it's not that if anything I mean for all I know for all I know the usage now talking about usage in terms of requests of SUS might might easily increase drastically because of agents hitting it hard.

36:54 I mean look at GitHub usage skyrocketed right all because of agents. [snorts] >> Yeah by the way look at GitHub's uptime. it sky plummeted due to AI agents. >> I mean to be to be fair to them kind of like I don't think that it was easy for anybody to predict this increase because >> oh it would as a company if you were prepared for this increase you were insane because that means that you overprovisioned everything like crazy.

37:34 >> Oh yeah. Oh yeah >> that would be insane. No, I we have to understand, yes, the uptime of GitHub is horrible over the last two years. I don't know anyone that would have been able to do a better job though in the industry with that large of a change in usage overnight basically. Not only that, but how many companies would not close the door to to free options if they had GitHub?

38:06 Because their cost must have skyrocketed as well. Kind of all those public public repos that are free. Oh my god, that must cost. >> Private repos are free also. >> And they're free as well. >> Yeah. >> Oh, I did not know that. Okay, there we go. Even worse. Speaking of actions costs you money when you're in a private repo above a certain whatever. Yeah, but the repo is free.

38:35 >> There we go. I mean, imagine the cost that that incures the increase in the cost. >> Oh yeah. Oh yeah. It it's absolutely ins. By the way, that's why like I don't know if you've seen this whole like news saga that's been going around with Codeberg, but who's like you know open- source whatever the Forge Go guys and whatever you know the broke off from Gy kind of you know whatever another you know git open source whatever free open source solution whatever they also have a SAS they just changed and they're like one

39:02 of the ones that a bunch of projects have been moving to from GitHub when they've been getting mad at GitHub they just changed their terms and policies were voted on I think it this week or last week. Um, no vibe coded applications are allowed to be stored on codeberg >> and it's against their terms of usage saying code that is majority written by LLMs even like cloud code with you mentoring it and whatever.

39:28 If the majority of a codebase was written by an LLM, it is not allowed to be stored on codeberg. And their answer to that like in the SAS version and their answer is listen like it's taking up way too much of you know the price that it's costing us to do this is insane just like no. >> Yeah that that that news just speaks volumes to the GitHub dilemma.

39:55 Nobody can nobody except Microsoft >> can actually allow that. Exactly. >> Okay. Uh Nick here, thanks a lot for the insights. Please can you elaborate on how I can start and gradually grow up to DevOps? Have you been doing it for years but not as a career? Um I think it's what Scott you mentioned earlier. It's all about understanding fundamentals. Higher level things are easy kind of like I Terraform because somebody mentioned it earlier.

40:29 Terraform is so bloody easy kind of the hard part is knowing knowing what to put into Terraform, right? >> Exactly. Like >> I always say if you can get a job even in help desk, right, at a large company at a help desk, you start there, you move them to the IT team, right? after you go there to the help desk and yeah when you're at the help desk get claude codecs whatever it is and write some applications that you know whatever that automate what you're doing in the help desk that help people that whatever it is and

41:03 whatever show that you are you know an innovator then from there get the company to promote you into the IT team right um from the IT department from ops and all of that you work there you learn the fundamentals You get down, you understand what a server is, you understand how Ethernet works, you understand traffic, you understand networking, you understand these things.

41:27 You're actually working with your hands and getting dirty with these things >> and then you move to DevOps from there. >> I'm going to use help desk. You mentioned this example, right? Uh if you do help desk, apart from doing the the actual job, you can be building stuff that helps you in that job. freeze your time for something else and at the same time you you learn through it right kind of oh I can build an application that will help me with those types of situations right >> and uh then it grows and you say okay

42:04 now actually I can run this in kubernetes I'm inventing lambda doesn't matter what whatever it is right you can just be learning by improving your own work conditions >> exactly >> and that that because That's at the end same same work every we are all doing the same thing. We are figuring out ways to improve something. [snorts] >> Yeah. And also I will say people need to use the free tiers that come from the clouds to their advantage when they're entering into the ecosystem.

42:35 Oh yeah. You can create an account on AWS and you can create EKS clusters. You can create ECS Fargate garbage. Uh you can create lambda functions. You can do all of these things. Try it out. Spin up a blog on a Lambda function with S3 and CloudFront. What just to do, right? And spin these things up within the free tier. Test things out. Try things. Play with things.

43:07 You can set up an AWS, a Google, an Azure account, get free tier every month. You get X amount of money. Whatever it is, just play with things. learn with your hands on. And if you're smart with how you do it, you can get so much for free from these clouds using their free tiers. [snorts] >> Could be just uh cloud nomad. Go through all of them. All of them.

43:34 >> Exactly. >> Almost all of them offer. >> Use ov cloud, use vulture, use whatever. It doesn't matter what cloud it is. Use a cloud. It has a free tier. Sign up for it. What do you care? You know the good thing about that also is that if you switch from cloud to cloud free tiers that will force you to do your stuff in a way that is reproducible >> to the cloud >> agnostic to the cloud but also reproducible okay so I didn't create just cluster by clicking clicking clicking I created a cluster using whichever tools and

44:03 those tools actually with a small rewrite can do the same thing over there and over there >> exactly because I used our because I use flux cd if I'm smart or if I'm less smart, I use Argo CD to deploy the applications onto my cluster. So, okay, fine. I need to change the Terraform module that deploys an AKS cluster instead of EKS. But once it's there and I've deployed the Helm chart using the Terraform, Helm release, whatever that deployed, you know, Flux CD onto the cluster.

44:32 Boom. Now everything else continues to work because I've abstracted away 90% of it away from the cloud, right? So I'm just changing that infra level right like and whether it's terraform crossplane palumi I'm not getting into that now also but exactly >> so what do you think of AI infrastructure roles and is it future proof >> two questions I mean AI infrastructure roles assuming that you mean kind of GPUs and inference and all that jazz is in high demand now high demand Now future proof I think that everything is

45:14 equally future proof because all those roles are 90% the same as any other role kind of a infrastructure role is only 10% different. I'm I'm throwing random percentage here whatever the percent is from any other infrastructure role just to be 100% clear and transparent. So it's not this is something completely different. is the same and by being very similar there are some details that are very different but being very similar it's future proof like anything else because tomorrow it will be something else tomorrow we

45:50 will I don't know do something else it all builds on top of each other it's all about those fundamentals that Scott mentioned so >> but it's it pays a lot now >> AI infrastructure role sounds like the biggest BS way of adding an extra zero to the end of a salary over any other infrastructure role just because it has the name AI in it. In the end, what are we talking about?

46:13 The fact that you know how to plug in a GPU to a PCI card, PCIe slot instead of plugging in a, you know, a network card. Um, or the fact that you know how to hook up a Melanox 400 gig backbone network instead of a 10 gig nick. Oh, oops. They plug into the same thing in the back end. Like what are we actually talking about of AI infrastructure role now?

46:34 Like that's where so many of these things everyone talks to me about like AI apps like oh you mean an application that runs with CPU and RAM oh it also has a GPU but it runs as a container like everything else inside of my Kubernetes cluster so there's nothing different um like >> there are some different things but >> yeah I if it's a inference it's different if it's an actual application that's just calling an application >> if you talk about agents agents are exactly the Same.

47:05 I was now referring to inference. >> Exactly. Inference is one thing. How many people are running inference themselves? >> Not many. >> Most people are still getting hosted inference whether it's inference on bedrock >> or Azure foundry or Google vertex or whatever at which point it's a Terraform resource or it's coming from one of the frontier labs themselves.

47:29 the majority of people are not running open weight models and actually doing inference themselves. um those that are again it's a bit different but it's 99% the same thing as running another application um as running any other distributed application let's say um that's why all of the AI stuff all these AI specific roles and whatever unless you're talking about AI architects right and whatever and things like that and actual data scientists the vast majority of these things I it just hurts me to see AI engineers and AI

48:10 infrastructure roles and all that. Now, if you can get the role, awesome because basically you are an infrastructure role and you're going to get an extra zero at the end of your salary over a typical infrastructure role. So, you go and do it. It just pains me that the industry is at this level where we just come up with random roles. AI ops and everything ops that add any acronym to ops at the end and you know we have just you know butchered the industry in terms of titles.

48:43 Again the same uh model plane where is it going? Any production yet? Adoption yet? Uh where is it going? Uh it's getting into now the focus is to add all the providers and um and a big focus on uh networking part of the story so that kind of it can reliably work across clusters um and do more complicated algorithms. Uh any adoption production adoption yet as far as I know?

49:14 No. It was released like two months ago. >> A month ago, a month ago, >> something like that. Yeah, I would be I mean I don't know it's open source. I cannot guarantee that there is no production but I would be surprised at this point yet. Right now hopefully some companies are trying it out but nothing goes to production first few months. Uh I mean first few months of something that is released early.

49:40 Some projects work for a long long time before they release it to public and then yeah but this is this is kind of >> basically it was released a month before after it started. >> Uh I love what you mentioned about starting with helpless though in Africa we don't have these types of companies. Thanks again for the tip on using free tiers. I will I'm going to start that.

50:05 Yeah, I mean, yeah, Africa, I don't know situation to be honest, but uh I always recommend people kind of if you can try to find some non-local work. There's a lot of remote work and the need for it. So, maybe check that out. >> Yeah. Uh how to learn AI infrastructure? Setting up home cloud labs literally seems impractical in today's time. We practice things like distributed inference and training with their DMA infinite bad touch.

50:37 So this is definitely not talking about agent Scott, right? This is definitely about inference. >> Yeah, setting up RDMA actual hardware all of that for like these AI inference farms and whatever. The answer is unfortunately unless you're a millionaire, good luck. Uh the real answer is learn learn read read read understand the technology from a from a theoretical level and then go to a job where they actually you know [snorts] have the money and have the stuff and you do it like there's no way to learn on these things

51:14 that cost millions and millions of dollars to set up these AI inference farms. you're not going to be able to actually, you know, set that up. And a cloud lab, you're not going to be dealing with RDMA, uh, and all of that. Like that's, I mean, you can, you can get >> part of the way on your own, not without budget, but with limited budget, you can do it.

51:39 You can some principles are different, but some are the same between large and smaller models. It's definitely not the same thing running multi-GPU Kimmy as uh small model but still you you can you can get part of the way with on a limited budget not no budget >> no but what we're saying here of like distributed inference across multime and all of that >> and train and training with RDMA and whatever like good luck like unless you have a lot of money Okay.

52:15 Uh, hi. Micros composition has become huge with different pipeline steps for provisioning private AKS cluster and other cluster rec resources key volt storage accounts as well. Uh, I'm not this is more like >> it's a continuation two messages later. >> No, but it's the same, right? >> No, it actually ends of is there a way to split? Is there a way to split it up?

52:44 You can. So a composition when it runs is a Kubernetes resource just like any other deployment service what's not right. You can separate it into logical groups groups hopefully reusable and then include composition inside of composition just like any other resource. >> Well hold on to include an XR not you can't include a composition within a composition.

53:07 You can include an XR in >> XR XR XR. Sorry. Sorry. Yeah, that's what I meant. Yeah. Yeah. Exactly. >> Right. So, there's a few ways there's a few ways you can go about this, right? You can do what Victor is saying, right, of build smaller building blocks, right? So, you have your my cluster resource, you have your network stack resource, you have your resource group re uh resource, you have your key vault resource that has 20 resources within it to create that key vault and everything that's needed.

53:38 But then all you're doing in your my cluster is setting network cluster resource group key vault whatever and you have 10 resources in there right so then it creates that tree the other way you can do it if you if it's not reusable >> right and your issue is just the size of the composition that's where you should be looking at the new crossplane projects and the new devx things where you actually write your compositions logic not in line, but you write actual custom functions that run as containers and the logic is

54:16 within that container. And then all of the logic is not inside of the composition. The composition just points to a running function that has your composition's logic in it. And then you don't have any of those limitations of like how big that file is >> because you know if it's not about reusability in my example uh what if you treat it as code your compositions no matter how big they are they will never reach the size bigger than a tiny app right in in the terms of size lines of code.

54:50 Yeah, >> they're never going I mean put everything in a single composition and that's still a miniature app in terms of lines of code. It's nothing >> as long as you treat it like code and not as one big YAML then you're >> exactly exactly >> uh is it easy to get job abroad as DevOps infest compared to others like data engineering back end? Uh, I'm guessing I mean >> I I don't know what abroad means and all of that.

55:22 Um, again, infra can be harder to get sometimes than other jobs as a remote position because a lot of times infra positions they want you near the data center, right? If we're talking about like onrem and dealing with actual infra things, right? That's a bit more challenging. Um, but things like DevOps, data engineering, backend, things like that are pretty easy because again, if it's not something that you're dealing with actual physical hardware, then no one really cares if you're local or remote in a lot of

55:59 companies, which I, by the way, I hate. I like people coming into the office. I think that it creates a good uh ecosystem. I think that when you actually know the people you're working with and all of that and you see them in the office, I think there's something good about that interpersonal things. Um, to a lot of levels, there's a reason I come into the office basically every day.

56:22 >> Um, but you know, a lot of companies do support remote work. [snorts] >> I agree on everything you said about remote work and I'm still not going to I'm still going to stay in my at my home. Uh I mean my my office is few thousand miles away. So anyways, >> exactly >> the you know the advantage it really depends a lot on location and the company right uh like your case it's next next to where you live your colleagues are next to where you live that's fine uh you could be working in a more global company like I am in

57:01 which case kind of the closest one is few hundred kilometers no thousand anyways Um yeah, I need to install EKS cluster. Would you recommend Terraform? That depends what for. If it's kind of like I want to play with the EKS AKS cluster. I thought it's EKS. AKS. Um >> AKS Azure. Yeah. >> Yeah. Azure. Yeah. Yeah. Uh uh I mean if it's something simple better than crossplay >> again.

57:35 If it's simple or whatever do Terraform. If it's something simple use CLI. I was about to say easy a CLI. Exactly. I I was looking because I thought it's AKS and I was about to say AKS cuttle and couldn't find the equivalent. Yeah. A >> exactly. So use a CLI whatever. If it's if it's something just to create and toss whatever a CLI works. If you're looking at actually building something that is going to be around for some time but again it's something small as side thing.

58:05 Great. Use Terraform. That's awesome. You know whatever. There's modules out there. Use Terraform. create the you know cluster all of that if it's something that you're planning on building multiclusters and this for production and you want to really you know whatever build out it you know where you can create multiple clusters and all that that's when I would start looking at something like crossplane >> okay how to handle legacy terapform stacks with github like pattern I I so this is not me bragging about uh

58:37 complaining about terraform just to be It's just that I think that different tools and different formats are built for different things. Terraform is not built to be reconciled continuously. Uh and I don't think that it fits into GitHub's like pattern. >> I know you can do it. >> Okay. Okay. So, I disagree with you, but I have a caveat. So I do not believe you should be using Kubernetes controller like what flux built back in the day or weave works built in not flux.

59:09 part of the flux project, but the weave works uh flux plugin for you know open tofu and terraform whatever the tf controller or things that who else built like a bunch of people built them also the guys from open application model uh whatever that uh cube built one also right and all that no I do not believe that that makes sense because you shouldn't just be running ter and by the way you shouldn't be using the crossbane terraform or crossbane open tofu providers either okay so no However, you can do GitOps in the

59:40 sense of Terraform Cloud with its with its source control integration, Atlantis, uh, N0ero, all of these tools that do GitOps. Now, what is my caveat to all of that? And this is where I have my major issue with a lot of these tools is that they do partial GitOps. Now, you want to go down the GitOps approach. That means you get rid of Terraform plan where you need to approve in order to apply it.

01:00:09 The second I push it to git, it gets applied. If you follow GitOps principles, what is in git is what happens. So that means that you need to say I push to git terraform apply-auto approve. >> If you're willing to do that, then perfect. Go with a githops like pattern for Terraform. But if you're not willing to do that, then don't do a GitHub style uh approach.

01:00:40 You can't have it both ways. I look, if you do auto approve, which I agree that kind of you should be doing that if you want GitHubs, the state is going to bite you in the ass every single day. Well, unless >> unless you actually do what I like, >> which is you have branch protection on main, right? I hope so because you, you know, work with standards, right?

01:01:08 >> You open a PR. What happens when you open the PR, >> a GitHub action or whatever runs Terraform plan >> and reports back the plan to you on the PR. >> You then decide, >> am I going to merge or not? The second that you clicked merge to main terraform ply-auto approve >> but the PR is your gate not terraform approvals down the road meaning what's in main continuously reconciled um if you do that then I'm fine with doing version control system based terraform and all of that and I think it's fine uh with githops like

01:01:48 patterns >> with that >> okay four minutes because I really need to finish at the top of the hour. Uh Nick again, you mentioned learning or working remotely. Can you please guide me how where to start? Perhaps someone could mentor me on this. Uh where to start? I always recommend open source projects if you have time. Uh kind of because that's how people because a lot of that works through connections to be honest.

01:02:17 I mean that doesn't have to be connection. you could just be applying to jobs and see what lands. That's one option. Uh longer term option is getting those connections and you get those connections through people you know and you get to know people through open source. Uh that's that's what I would say just it's not for the next week fix just to be clear.

01:02:41 If you want something next week or next couple of months that's kind of just apply to anything you can get your hands on. >> Yeah. And I unfortunately I have very limited time. I cannot accept mentorships. Sorry for that. >> Yeah. >> Um >> same but good luck. >> The other one I meant overseas jobs. I think that Scott's comment was Yeah, but overseas of what [laughter] >> exactly >> overseas of China, overseas of US, overseas of Europe [laughter] can be many different things.

01:03:16 And the last one which is two minutes away. Cool. Um I don't know what this means. >> Can you read that Vector? >> I cannot read for the love. The only thing I understand is it's left to right. So it's not Arabic. >> It must be Israeli. Right. >> Hebrew right to write left to right. Exactly. >> Left to right. >> We write right to left. English is left to right.

01:03:40 Right. Um Hebrew and Arabic are both right to left. This is Shabbat Shabbaky. Happy Sabbath. whatever the Jewish, you know, weekend uh brother basically, you know, whatever. >> Wait, is this left to right? >> No, this is to left. Right to left. >> But exclamation mark is at the beginning. Then >> it's because Streamyard and YouTube don't know how to deal with uh punctuation, so it always messes things up and puts it on the wrong side.

01:04:09 >> Ah, okay. I concluded it's left to right because of the exclamation mark. Okay. It's because of LTR, whatever, and all these terrible things that none of these apps know how to format Hebrew and Arabic that way. >> Uh, >> okay. >> But yes, >> thanks a lot for the chat. Really learned a lot. Thank you for attending. I need to stop it here. >> Thank you guys. >> Thank you everybody. Cheers. >> Cheers.