ShipTalk Podcast - Always Stay Curious - DevOps Leadership - Frank at DataStax

In this episode of ShipTalk, we talk to Frank Moley, a Senior Manager of platform engineering at DataStax.  Having the ability to respect multiple points of views and opinions is critical in modern engineering roles. We are the sum total of our experiences, and opinions are usually formed by the tangential thoughts and opinions of others. In this episode, learn from Frank as he evolved in roles from software engineering to DevOps/platform engineering leadership.

Transcript

Ravi Lachhman  00:07
Hey everybody, welcome back to another episode of ShipTalk. Very excited today to be talking to one of our good buddies, Frank Moley, who’s an engineering manager at DataStax. Frank, thanks for coming on.

Frank Moley  00:18
Absolutely, absolutely. I love doing stuff with Harness. This is a great opportunity to chit chat a little more.

Ravi Lachhman  00:27
So it’s funny that Frank and I actually have fairly similar backgrounds. We were having kind of a pre-conversation before the podcast that, you know, Frank and I kind of have risen up through the Java engineering days. But Frank, maybe you can tell the listeners a little bit about your background for those who don’t know you.

Frank Moley  00:45
Sure. So I’ll give you a great tidbit. My first degree is in microbiology— it’s not even in computer science. I had illusions of going to go solve the great diseases of the world, [then] found out you couldn’t really make any money. So computers are my passion, and decided to go into computers. My first job though actually used my degree to some extent, my micro degree, because I worked for a healthcare software company, actually doing a lot of back end C services, Visual Basic code. Then they started this language, Java, and I was quickly enticed to be part of that team. I got to rewrite a lot of legacy systems into Java, and we were targeting WebSphere, which had its fun days— back in the day J2EE dealing with WebSphere, and its unique class loading algorithms that were different than everywhere else. And, you know, big iron servers that brought everything with it.

And it was an interesting ride. I learned a ton. I learned things that I didn’t know, like when I was working with C server development, I’d write a memory map and just “okay, whatever, there’s memory on the server.” And then I started dealing with Java, and it’s like, oh, now I got to deal with this heap space, and did I have too much stuff running on the app server, do I need to spin up more instances… the more I got into it, the deeper and deeper I would get into the underlying technology running out. And then I got introduced to this relatively new framework at the time called Spring. I was writing some API’s using like Apache CXF, and stuff, and managing all of my own dependencies and object creation, and of course, having to deal with the heap and everything that comes with that. And there’s this framework that I found called Spring, and I started getting involved. And it was my first, really, introduction to open source. I quickly found that passion was Spring. I didn’t know I even had it, like it was everything I loved about Java— and everything that I hated about Java, I didn’t have to deal with because of it.

As Spring grew, and the industry kind of moved away from big iron servers— I don’t know, they still exist, I mean, you still have WebSphere, you still got WebLogic. But we started doing stuff with Tomcat and lightweight containers. And now all of a sudden, I could run an app in Tomcat and spin up like 30 commodity boxes and get more performance. I came with one big box, and now all of a sudden, the world was changing a little bit, right? It wasn’t about how much money were we gonna pipe into Oracle or BEA at the time— and an Oracle or IBM, it was, well, “I can just go run some Dells that are sitting in the IT help desk, and I can run a server farm on them and run a bunch of web apps.” And it just was an awesome concept, the commodity market. What I found was that the more of that I was doing, was sort of building this framework for where I got to— where I’m at today, where I’m building infrastructure and platforms and all of this because, slowly, as we tried to reduce costs and simplify overhead, we’ve started running the servers ourselves and running smaller instances and more of them. It’s the CloudBees model, right? Instead of throwing one big Jenkins, let’s throw hundreds of Jenkins at the problem and run it across multiple. So that’s sort of just continuing to set this framework.

And then I went to another company, and they were running in a single data center, a global company doing a ton of business. The IT department was sort of the hub of everything, and I got this opportunity to help them do two major things— and that’s really, in my opinion was the defining moment of my career. The two biggest projects that we solved was, one, I helped migrate them off of J2EE onto pure Spring framework. We were building microservices before microservices were the coin term. Like, we had these monolithic web services that we were starting to build into little domain-driven components. And of course, that breeds microservices. And the other big thing that we got to do was, we left the single data center model and went to multiple private data centers, globally distributed, all running on containerization framework.

[We were] doing all of this with Pivotal tools and Cloud Foundry and running Spring Boot apps everywhere. I’ll never forget. So Rob Winch is the Spring Security lead; I started a local user group on Spring and Java, here in Kansas City. And he put a tweet out and it was this little snippet of groovy code that was a complete Hello World web service using Spring Boot and the CLI tools for Spring Boot. And it was like, everything up to this point is right here. It’s like, “I can now have all of my Java code, and never worry about the web server.” And never worrying about a process running on an application or running on a server; an application server was bringing it with it. So it was kind of a cool evolution of not only my career, but really shaping how I look at development as a whole.

Ravi Lachhman  06:44
Yeah, that’s perfect. I think the aha moment for me, for that piece, was similar background, like I did a lot of JEE and then shifting gears like “that was right.” I was part of that write-once, run-everywhere type of mantra, so I was a little bit after we were taught. So tying, let’s say the most of the web servers I’ve worked on, were up at that point, it was like WAS, WebSphere Application Server. It was right at the cusp of them starting to be more generic, like “yeah, we could run it pretty much at any, any processor, you don’t need a big iron.” So as I started building more applications— you hit the nail on the head— our deployable units became smaller, right. Or we said, “No, we want, we need more because we need to have a cluster” or “we need more because of the scale, we need to have this.” The reason wasn’t really that many people focusing on let’s say, like web server, ND, the cluster deployment, or the cell deployment. Or when I started going into JBoss, it was like a domain controller and was like “Oh, yeah, like, let me help architect that.” And so that was like that bridge between application development and application infrastructure. And now, if you fast forward to 2020, a lot of we’re deploying is with containerization. And we’re deploying a lot all the time. Right?

Frank Moley  08:01
All day, every day.

Ravi Lachhman  08:02
All day, every day. Yeah, it’s a good play, what you need immediately and keep scaling until it meets your needs. And that’s where that path [is] kind of a criss-cross; so awesome. You know, how I would say, if you’re gonna talk to Frank without knowing Frank… like, “you know what, Frank, you’re a platform engineer, but he has a very strong background in software development.” If folks are in, let’s say, software development, or looking to kind of transfer into that, I think that’s a really good path. Like, “hey, take a look at where you actually have to deploy.” And so, this leads me to another question. Let’s pretend we’re in 2020, we still are, for the modern software engineer, the application infrastructure do they have to know like, how important that so when you and I started, like we knew where to point to WAS, we built two WAS, we knew where to point to WAS. I mean, all our web descriptor is going to WAS, but then that starts slowly fading away to oblivion to some point, but give it a modern sec, like what do you think: how close does an engineer half the be to app infrastructure today?

Frank Moley  09:08
So my answer will be controversial, but here’s how I look at it. There will always be a place in this industry for developers who simply want to write code. I don’t think that’s ever going away. And I think the birth of code schools and boot camps is just proof of that, that we will always have people that are fulfilled in their daily routine by just writing code and sending it out for someone else to manage. But I think there’s another class of developers— and maybe we’re fewer and further between. But there is a class of developers that is never satisfied with just writing code. Because there’s always more. I’ve got people on my team that run Kubernetes clusters in their basement, on, you know, eBay-bought servers that they found used; there are people that will always want to get deeper.

And I think that that’s how things like Kubernetes become what they are, it’s because people like me—and I’m gonna put myself in that group, right or wrong— always want more. I can write code all day long. And I can be happy writing code. In fact, the more code I write, the happier I am. So I don’t get to write a lot of them anymore, but I am happy writing code. But I tell you what, when I can write a script, that provisions infrastructure, and manage the infrastructure and lays Kubernetes on top of it, and then I deploy my applications to it— that’s like, okay, not only have I baked the cake and iced the cake, but eating the cake too. And it’s like having this entire fulfillment of everything. Like, I have to know how CPU is provisioned in the cloud infrastructure, as much as I’ve got to know how heap space is allocated in a job application. And that’s a lot, I mean, that’s a lot of process. And it’s a lot of problems to solve, because the deeper you get, and the deeper that systems that goes, the more troubleshooting you have to do, which is why it will never be for everybody.

I wish it was— it would make my life as a hiring manager a heck of a lot easier. Because quite honestly, it is hard finding somebody who is really good at infrastructure, and really good at writing code. Finding one of those two is, is “relatively” easy. And I say that relatively in air quotes. But you know, there are people that are really, really good at infrastructure. I mean, I’ve worked with some top notch Unix guys, that know infrastructure like there’s no tomorrow. But when it comes to writing code, if it’s anything beyond a script, they just get lost. And I’ve known guys, and actually a lot of really good women— I use the term guys and that’s not appropriate, so I apologize— but a lot of really good quality women, men, everybody that write really, really good applications and really good programs, but they don’t know anything about the infrastructure. So finding that mix is hard. But I think if it’s something if you’re a tinkerer, or if you’re somebody who really likes getting into it, there’s no better way to be fulfilled in your career than building the entire thing from soup to nuts and going out.

Ravi Lachhman  12:45
That sounds really interesting. Kind recapping— you have a very pragmatic answer, there’ll be some people who are shielded from the “hey, I just an external party that has to run when I write” versus some folks who are more, let’s say, inquisitive. I think from my perspective, it’s kind of a split-brain problem; now throughout my years of career, I can finally admit there’s two sides of the brain I have to use as a software engineer. My purpose up until now would be iteration, right. I’m given that I’m not going to get it right the first time. And so like, let’s say the last bit over the last couple years, I used to, up until the last couple years, butt heads with the infrastructure people all the time— like constantly butting heads with them. Because all the time through the infra, there’s no sandbox. You have to get it right. Like there, that’s production. There’s no iteration, versus you and I running something in Spring Boot, if you will see me rebuild it like 60 times before I get it. Yes, it’s like these prod boxes. Nah, you don’t get 60 chances to get it right. Well, one of that’s kind of shifting with infrastructure as a code, right?

Like, I’ve been noticing that I’m just kind of touching on one piece that you mentioned there, that it’s there are two separate sides of it. But maybe you can elaborate on [how] as you became more senior of an engineer, that was a problem— you have to focus and this is going to echo into something called a full lifecycle developer. So when I was actively in engineering, as it became more senior in principle, my type of metrics change. It wasn’t so much as completing the feature. It was more “is it in production?” And there’s a lot of in-between. You’ll communicate and having some sort of snapshot versus is it running? So let’s talk about that. Why don’t we talk about what’s the difference between building it and getting it into production?

Frank Moley  14:44
Yeah, and so before I get there, there’s one thing you talked about. And I want to echo this because it was very, very important. And it still is important. So I build platforms today. I build systems that not only run my code, but other people’s code. So I still have to get it right in depth. Like, I still have to have the infrastructure part correct, because somebody else needs that to write code. So I do think infrastructure code has helped a lot of that. But, you know, when it comes to this path of moving this way, I think it really comes down to “how deep do you want to get, and do you have a capacity to do it?” Because some people will get into infrastructure and just not like it. I mean, it’s monotonous at times, quite frankly— I mean, there’s only so many ways you can set up a routing policy on a network. And there’s just not a lot of variation. And that’s why infrastructure code works so well, right? It’s because you get it right. And then you can deploy it everywhere, until you get it wrong, and you’ve got to go back and start over. But I do think that a love of that has to sort of be not bred but learned. You have to learn to love the infrastructure, in order to get into this full stack.

Well, and I think for me, or full lifecycle as you put it, I think for me, it was just a desire to be more valuable to the organization as I moved up that caused me to expand my depth of knowledge. And really, the biggest part of that was at one of my previous employers. My master’s thesis was written on securing data in a public cloud system, PII data, and this was pre-GDPR. And they asked me to become the security architect. So in addition to all of my other roles that I was doing, they asked me to be responsible for our security program. Well, it’s very, very quickly that you realize that you can’t just fix security problems in code— you have to understand the infrastructure. And you’ve got to understand network topology, and you’ve got to understand what components are running on those boxes. So why do you use Alpine Linux for a Docker image? You know, a lot of developers are like just copied it from the other Docker image. And that’s why I used it.

Ravi Lachhman  17:28
That’s exactly what I do.

Frank Moley  17:30
And that’s fine, right? I mean, somebody’s already figured it out, why do you need to? But for me, I want to know why like— I’ll take you back to Hibernate, I know you’ve used Hibernate here. I was never satisfied with Hibernate. Until I actually got in and understood how Hibernate works. And I think for me, that was just maturing, how I matured in this industry. And I think that’s how I got into this full lifecycle type mode; I was never satisfied just writing the code, I wanted to know how the underlying bits worked. And then as I learned how the underlying bits worked, I wanted to know how the JVM worked, or the interpreter files in Python. And then, as I learned how that worked, I wanted to learn how the operating system worked. And then as I learned how the operating system work, I’m going to learn how the CPU work, right? So for me, it was just this desire to always get deeper. And I’ve had quite a bit of hardware training in all of my degrees. And unfortunately, I have a lot of them. Or fortunately, I did a lot of hardware work, I did a lot of servers and networking.

It kind of was not only [could] I write code, but I could understand route tables and routing algorithms. So for me to become more valuable to my organization. I felt the pull to know more. And it allowed me to spread across multiple fields. And then I can answer more and more people’s questions. And of course, once you start having those questions and the sessions, you get asked more questions, you got brought into more discussions, and then you learn more. And it just kind of breeds this sort of almost growing expanse of knowing more, knowing more, knowing more, and you get asked for and you know more.

And that’s really how it has grown in my career. And it really set the ground for what I became. I wouldn’t be who I was today, if I hadn’t taken those steps. But again, I’m gonna go back to this because I think it’s important. You know, a lot of new developers look at senior developers as like unapproachable. They don’t understand the world, the way we look at it as a new developer. And I think it’s important for us to remember that we were once new developers as well. And it can be very intimidating, looking at the amount of stuff I managed today when I was writing my first bug fix. So if it’s something that fits you, you will grow into it naturally, if you put forth the effort.

Now it requires a lot of study, a lot of reading. The only language I was formally trained in was C++; the only operating system I was formally trained in was Windows. You have to do the rest of it yourself. I was never taught Kubernetes, I was never taught Cloud Foundry; I’ve learned them. So that was for me what worked. And I think that that’s also a sign of growing into a senior, is just a desire to learn more. Now, you can become really deep on algorithms and learn how to fine tune the JVM and get the most out of your sorting list. Or you can go the other way and go learn about the hardware. So again, there’s a place for everybody. And not everybody has to follow the path that I took. But there are those that do, and you will find that there’s a lot of rewarding work, especially in cloud today.

Ravi Lachhman  20:56
Awesome. Yeah, that’s great to know. I feel like this is my therapy with Frank. Why do I do so much platform stuff? Bow? I’ll give you kind of a more intrinsic question. And in the couple of DevOps Days that I kind of, like float around on, these are always very intrinsic questions. It’s about it’s kind of a core job of a platform engineer. But it’s something like it’s hard to get, right. So as a platform engineering leader, it’s when you’re building a platform to service your internal customers, it’s all about opinion— like how much of your opinion do you put into it versus trying to pull out? What are the requirements for internal customers? So for example, let’s say (by going back to pretend when you engineer at your firm), it’s like “why did Frank make me go through all these DevSecOps checks or whatnot?” versus “Well, you need to do that.” So this is the gray hair beard versus the fresh cut. How do you balance that?  A difficult question that a lot of platform engineering leaders kind of deal with.

Frank Moley  21:59
Yeah, no, it’s a great question. It’s actually one that I talk about on interviews when I interview candidates, because I think that there’s two flaws that people make. And I’m going to address both of them here. So bear with me. The first one, you made the comment about my opinion. And I would be remiss if I ever said that it was my opinion. And the reason I say that is my opinions, if you want to call them that were formed by a lot of other people. And, you know, I’ve listened to Martin Fowler and read his stuff. And, you know, Uncle Bob, and Mark, Mark Richardson; I mean, I follow these people— Matt Stein. There’s a lot of people out there that have shaped my opinions. I don’t want to say I haven’t had an original thought, because I don’t think that that’s a fair statement. But everything is shaped by somebody else. So acknowledging that everything is shaped by somebody else, also will allow you to accept that you’re not going to have the right answers all the time. And this will take me back to the interview question.

One of the things that I stress heavily with potential candidates, and then those that become new hires is that everybody’s opinion matters. And the example I give is, you may not realize that that stating your opinion or asking your question is going to solve the problem until you see that question trigger a tangential thought— that then becomes the solution, right? So even if your opinion is not perfect and right, sharing it within the group allows you to maybe get onto a tangential path— that will be the right answer. So everything that we build as a team really is the team’s opinion, and we come to a consensus. Now, of course, at some point, you have to say “no more decision making, we’ve got to make it we got to move forward.” But the reality is, is all of that has gone through vetting. And you know, I still have my code reviewed, I still have my designs reviewed, I still have my opinions discussed. Just because I’m the senior manager on the team doesn’t make me perfect in any way. And I think that that’s an important thing for people to address— everybody’s opinion matters.

So coming back to the platform. Obviously, we have to operate right? We have to build a set of guiding principles and processes and procedures, and we definitely have an opinion on how those things work. But just like we strive in engineering software of the iterative, we also have to do the same with our platform. So, you know, a person new to the platform may come in and say “why did you guys do it this way? Why didn’t you do it this way?” We may go to make changes based on those opinions. So I think having an opinion, a well formed opinion, and sticking to it— being open to modifications when you need to. And again, it’s like every other problem in software. We decompose the problem into the smallest thing we can we solve it, and we don’t always solve it the right way.

So I really think— and this is why I’m gonna kind of get a little soapbox here, guys— I hear a lot of people talk about well, senior engineers are unapproachable, and junior engineers don’t know this or don’t know that, or, you know, whatever category we put people in. I think the problem isn’t that opinions are being made and decisions are being made. I think it’s we have forgotten how to disagree. You know, “disagree” has such a negative connotation. I think the ability to argue about technology and argue about solutions is how we all get better. So again, it goes back to that maturity of accepting that you’re going to know what you know, and you’re going to have a desire to learn what you want to learn. And that’s going to shape your career— forming opinions and arguing, and not being so headstrong that you don’t really do traveling to change. I think it’s really what breeds the proper culture.

I always give this final comment to new hires when I have my first meeting with them. At the end of the day, we have to remember, people are paying us to sit on a computer and tell the computer how to work. And most of us, at least, I hope can think of no better way to spend time than sitting on a computer. So you know, if you don’t take some joy in that, then all of this talk about growth and where you’re going to go is kind of moot— because at the end of the day, we’re getting paid to write computer programs. I mean, how cool is that?

Ravi Lachhman  27:27
Passion! Love it. Yeah, one excellent, excellent point you brought up there. It’s not an easy question to answer. Like, that was very pragmatic; we’re all a sum total of our experiences, right? And so what’s excellent about diversity and ideas is hearing multiple— and it takes people a little bit of time in their career to kind of be more open to it.

A very funny thing is I’m growing continue to grow as I get more senior in my career, is that just the amount of feedback you get. When I was an analyst, engineer, or a staff engineer, the only feedback I would get would be like a a code review every couple of weeks. And that’s it. I would have total free time in between, until we come up to a sprint review or something like that. Versus today, I get feedback probably every hour. Right? So it’s dealing with that, that compression— like, you’ll have to readjust course, listen to multiple people, like broker that— and that those are things that are you know, not taught in the textbook. It’s taught with time.

Frank Moley  28:34
There’s a couple things that I still say to this day that they don’t teach us in school, to your point here. And one of them is how to not take feedback personally. And I think at the end of the day, all of this conversation around growth and how you do your career, the best way to approach moving in depth knowledge, whether it’s the way I’ve done it, where it’s the full stack… I keep saying full stack, and I’m sorry, it’s not full, I look at what I do is full stack back-end development. But this full life cycle, right— whether it’s not or whether it’s just really deep in application code, or really deep and infrastructure, you have to be able to take feedback, because if you can’t take a code review and not take it personally, or an infrastructure design, or even a bug— I mean, we’ve all coded bugs. If you take that stuff personally, you’ll be this richer.

Ravi Lachhman  29:29
That’s a feature, Frank, not a bug. You know that!

Frank Moley  29:31
Well, yeah. Working is designed in the undocumented manual, right? Oh, yeah. But I do think that not taking things personally, and accepting that you don’t know all the answers. Because— I can share stories all day long. I have more junior engineers to me, some of whom I still work with, who have taught me stuff every single day that I’ve worked with them. I think if you’re willing to accept that you’re not gonna know everything, you’re gonna just be more open to learning new stuff. And again, it all kind of comes back to what do you want to learn. And if you want to learn it, you’ve got to set yourself up to do it. And one of those ways is accepting you don’t know everything. Because I don’t, and I don’t pretend to, and I don’t pretend that perfect designs. I mean, I tell a lot of people [that] if I look at code I wrote yesterday and I don’t see a problem with it, I’m not growing. And the same goes for infrastructure. The same goes for even how I managed people today. Like, you know, if I look at the way I communicated something to somebody, and it’s like, “I did that wrong”, then I’m growing. If I don’t see a problem with it, then I’m probably the one who has a problem.

Ravi Lachhman  30:50
Very elegantly put. Sometimes we’re our own biggest critics. But that’s how we grow and continue to evolve our ways of thought. I have one last question before we before we wrap up the podcast; and this is how I end every single podcast. Imagine you were walking down the street and you ran into a young Frank; you’re fresh out of university. What would be a few things that you would tell yourself, you know, face-to-face with young Frank, this green in the collar?

Frank Moley  31:26
Oh, that’s a good one. Don’t be so obstinate. Is that one of them?

Ravi Lachhman  31:33
You could say, yeah. Don’t go to jail, you know, stuff like that.

Frank Moley  31:37
No, I mean, that’s great. Because there’s a lot of those experiences. I think the one thing—and this is gonna sound kind of corny, but I don’t care because it’s the truth— [is] I would tell myself to listen to my dad more. So, and I’m going to share a little bit here; my Twitter handle @fpmoles. My dad called me FP since I can remember, but my name is Frank Peter. So is his, and he called me FP. So the FP part of my Twitter handle, my GitHub handle, my Slack usernames, all that, is an homage to my dad. And the other one is to my greatest mentor— he called me Moles, so it’s a concatenation of those two. But I do think when I left college, I wanted to be independent. I was the kid who moved out at 18. I was gonna blaze the world and do everything on my own. I didn’t listen to my dad enough. And I think if I had, I would have made fewer mistakes in my career.

And we’ve all made mistakes, right, but I’ll never forget one of the one of the worst ones I had— I actually erased the entire SVN history. Back when we had SVN, right, everybody’s got their RM dash f moments, that was mine. And I like immediately [was a] coward and didn’t own up to my mistake, which of course caused more work. It was nothing that couldn’t be repaired. But still, the fact was, I didn’t own up to it. And then I’ve always looked back on that, and said, if I’d only listened to my dad when he told me to own up to your mistakes, immediately, I would have saved myself a lot of pain and stress.

And there are numerous examples of that throughout my career, where I just didn’t listen to people who are wiser than me, and my dad was one of those. So obviously, I’ve got a really good relationship with my dad, so I like to bring that up whenever I can. But I think that would be the biggest one. And there been other mentors in my career, more professionally. I’ve been blessed to really work with some great great managers throughout all my career, who’ve really taught me a lot, including to this day. I mean, the guys who I work with, guys and girls I work with, whether they’re directly over me or adjacent leadership-wise… I mean, it’s just amazing what you can learn from other people’s experiences if you’re open to it.

Ravi Lachhman  34:09
Yeah, that’s perfect. And that was really kind of kind of hitting it, because owning up to your mistakes, it’s something important. I remember clearly, when I was dealing [with it] in my career, I wouldn’t want to be the guy who got fired, you know, but, we all make mistakes and the more senior you become, the more mistakes you make, right? Like instead, people are very empathetic on “hey, we’ve all been there.” But that was awesome, great homage to the father. You know, with that, I think we’re at the top of our time here at the ShipTalk podcast. Frank, thank you so much for being on with us today.

Frank Moley  34:45
Actually, I had a blast. And thanks for everything you guys do. You guys really do make my life easier every day. So thank you for that and look forward to continuing building out our relationship. Have a good one.