SCADA-as-a-service, appraisal mishaps, and time travel

We talk about the future of SCADA systems, a very unfortunate error in a Utah appraisal district, and differing movie-time-travel rules.

0:11 Patrick
Hey, guys, welcome to ZacCast. This is Patrick Lawler with my buddy Chad Janicek. What's up, Chad?
0:15 Chad
Hey, everyone. How you doing?
0:16 Patrick
Good. Good. Hey, man, I am gonna get us started today with a topic. Um, we have looked at the city of Hudson Oaks for a little while. Uh, just a couple days left in that job for me, but, uh, we'll continue on to help them with some pretty substantial projects. And one of the projects that we've looked at, which has been really interesting, is, uh, SCADA implementation. Uh, anybody knows anything about SCADA? It's a system that controls water and sewer systems, and it's used in pretty much every major city, small city, medium-sized city. Everybody has some type of automated SCADA system.
0:50 Chad
SCADA was, like, one of the first sort of Internet of Things implementations. So now we have Internet of Things on, like, fridges and ovens, and, you know, everything is connected to the internet. But, um, SCADA was sort of the first connected platform that cities used, um, to monitor things remotely.
1:11 Patrick
Yeah, I mean, if you're a city guy and you don't... or a city person in general, and you don't use SCADA now, um, Internet of Things, like, if this, then that, uh, the app that kinda says, "If something happens, then we'll have this happen," right? Is pretty much SCADA, right? So let's just talk a water system in general. I'm not gonna get super technical 'cause it can get really old.
1:34 Chad
Like, if this water tower gets to this level, then turn that pump on, and-
1:37 Patrick
That's correct. Yeah, water to-- say, your water tower is 150 foot tall, your water tower water level gets down to, like, 146, turn on a pump so that it will put water in the water tower to raise it from 146 to 148, right? That's how SCADA systems are built. So about three years ago, I started a process... Um, SCADA is controlled by a, a number of s- of companies that are called integrators, right? These are companies that are very well connected with the hardware side of things, but they all program in a SCADA language for just a few different programs, right? Uh, one, one of those is... And I'm not, I'm not calling out vendors 'cause they're terrible people, I'm just calling out vendors 'cause they're technology. Um, one of those is Wonderware. So Wonderware was, like, this system that's-- it's, it's prolific. It's pretty much been everywhere. Um, and it's a, a system that runs, like, on a, on an old-school Windows XP or previous computer, right? And you pay a significant amount of money to upgrade the software. Th- this is one of those areas where the market has not yet been disrupted by software as a service. It's, it's going, and I'm gonna talk about that. It's being disruptive right now. Uh, a- a- and so, uh, but Wonderware is a product that basically runs on a desktop computer, and your whole SCADA system goes into this one desktop computer or a server of some kind, but it really has a single point of failure, which in cities we try to get away from altogether, right? Lightning strikes that building, burns up that computer, your entire SCADA system fails, just goes away. And a lot of cities, surprisingly, are set up that way, uh, with this single point of failure system is there. So about three years ago, I was walking through my water facility and talking to my, my water director, who is, you know, a 30-year water director employee. He's really, really intelligent, he's taught me a lot. And I, I see this computer sitting over there in a corner, and, and I'm like: "What is that?" And he's like: "Oh, that's our SCADA computer." And, and literally, I'm like: "Well, that 1986-looking computer over there, that, you know, it's, it's like DOS on the window, is running our entire water and sewer system?" And he's like: "Yeah, that's, that's the system that runs Wonderware." And I'm like: "We gotta do something about that," three years ago. So I get on the phone, I start having conversations with all these different, uh, SCADA implementers. I get on the phone, obviously, with Wonderware, 'cause that's who we contract with now, and I just assume, like: "Oh, there's a cloud solution now." You know, we've got document cloud solutions. We-- obviously, we do sales tax cloud solutions and revenue solutions, a- and I'm like: "There's gotta be a cloud solution for this." Guy literally laughed at me on the phone when I called. I mean, I call and I'm like: "Hey, you know, looking for a cloud solution. What are you guys doing in the cloud?" "Oh, nobody's ever gonna do SCADA in the cloud, and it never happened." "Awesome. So how much is it gonna cost me to get a new, new Wonderware upgraded system for, you know, Windows 10," right? "What, what is that gonna cost us?" "$250,000, plus $30,000 a year in annual fees," right? Maintenance fees.
4:34 Chad
So it's like, that's like 2,500 bucks per household?
4:37 Patrick
Correct. Yeah, like, when you, when you ran the numbers, that's, that's how much it costs, right, for automation within a system. So, you know, of course, the first place I go in- is, "Okay, there's gotta be a better way." Three years later, I found a better way. I mean, that's, that's how long it took. Uh, and it was actually Brittany in, in our office... I, I just-- I said to Brittany, I'm like: "Hey, I've got a project for you. This is what we want. Go see if you can find it." And she happened to find a company that had worked in, um, logistics management, uh, like mobile logistics, like tracking trucks, how fast they're driving, you know, cameras on those vehicles, that type of stuff, right? Uh, a company called Samsara. And they had also gotten into SCADA systems in oil and gas industry. And, uh, they created, like, an online software-as-a-service, SCADA-driven module that's there. And, and I bring this up just because we, we start talking about, like, technologies that are moving into city government, and there's probably other vendors out there that are doing a little bit of what Samsara's doing. But I'm talking to these other cities, and, and, and a lot of them are talking to me like: "What are you doing?" And I'm like: "I'm taking a soft- software as a service for my SCADA system." "Wow! I, I can't..." Like, that's so foreign to everybody. But I just know five to 10 years down the road, every major city out there is gonna be-... in the, in the cloud with their SCADA system. Um, and, and we're moving that direction now. And this vendor has basically just developed a process that, that would've cost me $250,000 for the implementation, it's, it's gonna end up costing me about $40,000 in hardware and about $6,000 a year to maintain.
6:20 Chad
So hardware, you got sensors on, you know, different pieces of equipment.
6:23 Patrick
Yeah.
6:23 Chad
You've got switches that are turning pumps on and off.
6:26 Patrick
Correct.
6:26 Chad
You've got-
6:26 Patrick
Tank level sensors, switches on pumps, uh, you know, PSI sensors, uh, chlorination sensors. You know, the types of things that you need to run a water and sewer system, they're all connected in there. All Samsara did was take what, what are called, uh... They're called PLC devices, right? But they're basically IO devices. They bring in, uh, you know, uh, devices that tell something to do something.
6:49 Chad
Right, input, output.
6:50 Patrick
Input, output is what IO stands for. And, um, you know, they basically took that old school PLC that used to communicate through a radio signal, sometimes licensed, sometimes not, and they said, "You know what? We're gonna put a 4G LTE chip in it, or we're gonna allow it to be connected directly to fiber, or we're gonna allow it to be connected to some type of DSL phone line or whatever it may be, and we're, we're gonna put all of that information into the cloud, and then we're gonna build a really simple format, um, system, that's a software as a service system, so that everybody can build on their own, their SCADA system." So I can go in and totally customize my SCADA dashboard. Very similar to what we've built, you know, in, in Zack, where you can customize the front of our program for the city that you're in, right? Um, it is, it is very millennial-driven system that, just from a price standpoint, is gonna u- utterly devastate the market. I mean, I know cities that... There's a big vendor out there called Ignition that charges you, you know, $30,000, $40,000 a year, uh, for their software package, and cities have been going through the Ignition programming side for two to three years. They're, they're s- they're charging... They have vendors that are coming in, and they're spending millions of dollars for integration work just for the programming side. And, uh, these guys at Samsara, and probably a couple other vendors as well, have automated that whole, whole process. They've just simplified it. Still does all the same thing, it just, it does it on a scale level, and then in a software as a service model.
8:22 Chad
So how are they securing the, the internet connection from these different devices or these different sensors?
8:29 Patrick
So it's, you know, it's very similar to what we do, right? It's SSL encrypted behind a firewall, and it's, you know, it's a, it's a login system. Uh, it's, it's a whole lot safer than the, the standard protocol that's used by cities today to get to the access off of iPads. But... So let's talk about that real quick, because, uh... Sorry, go ahead.
8:47 Chad
Well, you're talking about your access as a user into the system to control things, right?
8:51 Patrick
Correct, yes.
8:51 Chad
So I'm talking about the actual, from their servers to your actual sensors. So you have a sensor that's reading tank levels, and it says it's 140 feet, and it's sending that information, instead of over radio-
9:06 Patrick
It's sending it over a 4G LTE connection, correct.
9:08 Chad
Right. So, so you gotta worry about that connection. But I guess I wonder, um... So this is probably an area where people are concerned about moving SCADA to the cloud.
9:17 Patrick
I mean, it's a, it's a major homeland security-type thing, as we would say in government, but-
9:21 Chad
Yeah.
9:22 Patrick
So-
9:22 Chad
You'll have to forgive me here, 'cause I don't-- I'm not as well-versed in the SCADA stuff.
9:26 Patrick
Mm-hmm.
9:26 Chad
So if you're using a radio band, is that sending it to, basically to your computer that is running the SCADA software?
9:34 Patrick
That's correct. And if you're smart enough in SCADA design, you can just, y- you could just-
9:38 Chad
Can you intercept the two?
9:39 Patrick
You can intercept it.
9:39 Chad
Is that radio signal also sending instructions back to the devices-
9:44 Patrick
It-
9:44 Chad
... to control them?
9:45 Patrick
The radio signal is... Yeah, they're going both ways.
9:47 Chad
Okay.
9:48 Patrick
So your signal is going both directions.
9:48 Chad
So you could theoretically intercept, like, sort of a Max Headroom intrusion type of situation?
9:53 Patrick
That's correct.
9:53 Chad
You're familiar with Max Headroom?
9:54 Patrick
Yes. Yes, yes.
9:54 Chad
Where they, uh, they intercepted the, uh... Like, this is back in the, like, late '80s, early '90s-
9:59 Patrick
Uh-huh
9:59 Chad
... I guess. And, uh, WGN got their signal intercepted, and, uh, someone was, like, pirating a, uh, a Max Headroom video.
10:08 Patrick
Uh-huh.
10:08 Chad
It had some interesting stuff on it, but-
10:10 Patrick
Yeah.
10:11 Chad
But yeah, I mean, if it's just radio signals, you could, you could inject yourself in the middle anyway.
10:15 Patrick
You could.
10:16 Chad
And the radio signals are not gonna be encrypted.
10:18 Patrick
Th- th- that's correct. I mean, the, the most famous hack of this was the United States government hacked the Iranian centrifuges, the SCADA system.
10:26 Chad
Was that Stuxnet?
10:27 Patrick
It was Stuxnet. You know, and basically they, they used that vulnerability in that system to hack it, which is what really opened everybody's eyes to, wow, SCADA systems are really vulnerable, right? Um, so, uh, I, I mean, the reality is, is that this is a very sensitive area of city operations. Um, you know, it's, it's a major criminal offense if you, if you interfere with the operations, 'cause we're talking about people's clean drinking water, right? And sewer. Um, and, and so, you know, I, I think we need to look at different ways to secure that data than what that traditional over-air system is. Uh, no different than people who, uh, started hacking into, um, the emergency sirens of cities. That happened, you know, like two years ago in kind of the Dallas-Fort Worth metroplex, there were a couple cities that had issues with that, where people figured out the radio transmission signal to, to set off those sirens, and they were setting sirens off all the time in the middle of the afternoon.
11:23 Chad
These sirens have, uh, like, radio frequencies that if you ping in the correct sequence-
11:28 Patrick
Correct, yeah
11:28 Chad
... you could, like, drive by or do it from remotely-
11:31 Patrick
Right
11:31 Chad
... to turn the sirens on, rather than having to go flip a switch to the-
11:35 Patrick
Yeah, and, and I don't know if anybody was this, like this level of nerd, this is probably one of the nerdiest things I've ever said on this show, but, uh, you know, when I was a kid, I had a, uh, like a really good... I don't even know if you call them walkie-talkies, but it was a radio. It was a CB radio. Uh, it was something I asked for for Christmas, and it was a really super high-end CB radio, and I would talk to people on the CB radio.
11:55 Chad
Like truckers?
11:56 Patrick
Like truckers. Yeah, I would talk to truckers and, and other people. So, I mean, as a, as a 16-year-old or 15-year-old kid, it would've been fun for me to try to figure out the signal to set off the storm sirens, right? So now everybody's gone out and spent hundreds of thousands, if not millions of dollars, to encrypt all of the storm siren-... coverage, you know, frequencies. So just, it just is what it is. I mean, there, there's a change in that market that's occurring in, in, in this, this cloud-based change. It's just interesting to me in technology standpoint, that cities in some areas have moved to the cloud and some areas haven't. Are the servers that we have at our, at our city facilities actually safer than the servers that maybe we have in the cloud that have multiple backups in multiple different locations?
12:42 Chad
Well, that's- I guess that's the big question, right? Um, and certainly if you're running, if you're running an old operating system, uh, as the server for your SCADA... I mean, if XP hasn't been updated in years-
12:58 Patrick
Uh, you can't get-
12:58 Chad
You can't get security fixes anymore
13:00 Patrick
... you can't get security fixes. That's correct.
13:01 Chad
So I guess you're kinda hoping that maybe, maybe that computer is just not connected to the internet, right? It's just, like, h- hooked up to the, the SCADA devices, but it's not online, so maybe you hoping that people won't hack into it-
13:13 Patrick
Yeah
13:14 Chad
... um, unless they have physical access to it. But, uh, there are still plenty of other security issues with running an old operating system.
13:21 Patrick
What I'm hoping is that the computer doesn't die before we get transferred over to the cloud. I mean, that's the, that's the bigger concern, right? But, uh, I, I, I think, y- you know, there are a lot of things that are moving this direction in cities. I just bring this up because I think it's really early in the process. You know, there are other things in cities that have moved to the cloud. Obviously, document management, um, you know, full data storage backups are- have pretty much almost every city has moved that to the cloud at this point, uh, mainly 'cause of crypto viruses and things like that. But, um, you know, this is just one of those areas I think we're probably five years maybe ahead of, of the rest of, of cities that are there. I don't know a city, uh, a single city in the state of Texas that has moved to the cloud, and there's a couple in California, 'cause that's where Samsara is based out of, that they've kinda demoed with. Um, but-
14:08 Chad
Well, part of the problem is lock-in, right?
14:10 Patrick
Correct.
14:11 Chad
I mean, you have to buy... If you have to buy new hardware-
14:13 Patrick
Yeah.
14:13 Chad
Um, it's one thing if you have a sensor set up that can communicate through some kinda protocol with multiple different software packages-
14:20 Patrick
Wh- which they do that. They can actually turn their, their IP device, uh, you know, for people that are, uh, super technical, uh, in, in SCADA, uh, they can actually take their, um, their on-site device and turn it into more of a bridge, right? So they can, they can use your existing, say, Siemens or SCADAPack, uh, or whatever, you know, facility you, you have on-
14:43 Chad
Connect the signals coming in, and then send it to their own system.
14:45 Patrick
And then send it to their own system, and it, it goes through that encrypted system that they have to keep those, those-
14:49 Chad
But you're still sending it through the radio.
14:51 Patrick
No- well, no, you're not. You're actually putting that... You're putting their IP device right next to the SCADAPack before it hits radio, and you're-
14:57 Chad
So how does it get from the sensor-
15:00 Patrick
You, you use a-
15:01 Chad
-to the-
15:01 Patrick
... use an RJ45, comes out of the SCADAPack device, and then goes directly in a Samsara device.
15:05 Chad
What's that?
15:06 Patrick
Samsara just bridges it, and then puts it on the 4G, 4G LTE.
15:09 Chad
What's an RJ45?
15:11 Patrick
Uh, so it's a... It's like a cat cable.
15:14 Chad
Okay.
15:14 Patrick
Yeah. I mean, it's, it's, it's like a computer cat cable. Sorry. I got a little technical there. I don't think I've ever been nerdier than you on the podcast. I just wanna point that out. So-
15:23 Chad
That's why I wanna talk about this.
15:24 Patrick
I know. I s- I kinda started my career in water and sewer and parks, and that type of stuff, so, uh, this has always been a, a huge interest to me. But, uh, I, I just see these guys... Uh, you know, sometimes a lot of time- because of what we've done with Zach, when I see technologies that are coming into government that I think are gonna be significantly disruptive, that are gonna grow really fast like we did, I- it just, it interests me, and, and this is one that just kinda really interested me.
15:47 Chad
Well, part of the problem that people have with this, A, is the lock-in. You have all the hardware. Unless, unless there's some kinda bridge, you have hardware you have to replace.
15:53 Patrick
Correct. Absolutely, yeah.
15:54 Chad
And you can't really do it piecemeal, or you're gonna have to operate under two systems, and no one wants to do that.
15:59 Patrick
Mm-hmm.
16:00 Chad
But, you know, there's also, I guess, a legitimate concern about something going wrong. But the truth is that, um, the platform is only as secure as the vendor makes it. So even if it's hosted in the cloud, it can still be secure or not secure.
16:16 Patrick
Yes.
16:16 Chad
Like, being physically lo- or not physically, but located in the cloud doesn't make it inherently less secure.
16:23 Patrick
It doesn't make it inherently less secure or make it inherently more secure.
16:26 Chad
Right.
16:26 Patrick
Right? You still have to be dependent on the vendor. Uh, whether it's on a computer that's, that's local, or it's on a computer in the cloud, if the software is a bad piece of software, it's gonna have an opening. I mean-
16:38 Chad
S- so what could go wrong? I mean, like, if... I guess with Stuxnet, they basically shut down all the centrifuges?
16:42 Patrick
Well, with Stuxnet, they actually overspun the centr- the centrifuges. That's, that's what they did.
16:46 Chad
Seems safe.
16:48 Patrick
Right. Yeah, right. Yeah. Uh, so, so they, uh, with Stuxnet, they actually communicated to the, the main platform. My understand- I, I read a bunch of articles on this, so I may get some of this wrong. But my understanding is that they, they basically showed the engineers on the industrial control system, the SCADA system that was running it, that the centrifuges were spinning at the speed that they were supposed to, and in fact, the centrifuges were overspinning, uh, which, which didn't allow for the uranium enrichment that they wanted, and it actually ruined- it set 'em back, you know, a couple of years in enrichment, so...
17:18 Chad
So I guess what... Outside of, like, a terrorism situation-
17:22 Patrick
Mm-hmm
17:22 Chad
... what would people do if they were to try to intrude into your SCADA? I mean, they're just gonna shut pumps off and-
17:29 Patrick
Shut pumps off
17:29 Chad
... like, play around, or what?
17:30 Patrick
Yeah, shut pumps off, you lose water pressure. Uh, you know, over-chlorinate. I mean, chlorination devices are on those systems as well. I mean, it, you know... Is it, is it non-recoverable? Uh, I mean, it could be. Like, if, if you run a major water treatment facility, and you over-pressurize the water treatment facility, I mean, you can get shut down for weeks. Uh, we've, we've seen that happen before. And so, um, you know, there, there could be, you know, really bad and, and painful things, but it's not, uh, it, it's not gonna shoot anything at anybody. You know, I mean, let's, let's, let's be, let's be fair and honest about that.
18:06 Chad
So talking about things that can get screwed up-
18:10 Patrick
Yeah, absolutely. Yeah
18:11 Chad
... I'm gonna have to switch gears here. This is a l- a couple of months old, this story, but it's been on our to-do list to talk about, and we just haven't really had the time to, so we're gonna make time today.
18:18 Patrick
Mm-hmm.
18:19 Chad
Um, there was a story a while ago about- I, I can't even hardly s- explain it, 'cause it's so ridiculous. The headline reads, "A Utah home was accidentally valued at more than $1 billion when a phone fell on the keyboard."
18:36 Patrick
It's ridiculous.
18:37 Chad
And it goes on to say, "And taxes will now have to rise to cover the mistake." First of all, let's just go background on the story.
18:42 Patrick
Yep.
18:43 Chad
So this is in Utah, and, uh, a house that should have been appraised for $300,000 was appraised for $987 million. So, uh, when it was identified months after, you know, rates had been adopted and budgets had been approved, and they realized, "Oh, crap, we're gonna be way under budget in our projections," um, the, the excuse was that a keyboard must have fallen... Or I'm sorry, a phone must have fallen on the keyboard, and it just j- jumbled a bunch of numbers, and then that's how this happened.
19:14 Patrick
And what's the separation in time between these two things? Like, who came up with this ridiculous excuse about a phone falling on a keyboard?
19:21 Chad
Uh, I think, like, the administrator or the assessor-
19:24 Patrick
Yes
19:24 Chad
... for the, for the county. But, um, yeah, I mean, at some point in, you know, during the appraisal process, this happened, and then things get certified, and no one caught the error, and then budgets got approved by the municipalities and the counties. Um, and then after the fact, they identified it. So I don't know what the timeline between when it was identified and when they came up with this BS excuse, but, um, I'm just thinking about someone sitting there on their computer, like, texting, and then a phone falls, and it pushes the keyboard, and then they don't look to see what it pushed and what, what they were editing.
19:57 Patrick
Yes.
19:57 Chad
Like, it's absurd. But-
20:00 Patrick
It's also absurd that there was no check or balance or QA-
20:02 Chad
Yeah, I just wonder-
20:03 Patrick
... QC or-
20:03 Chad
How do you-
20:04 Patrick
... anything like that on the, I mean-
20:05 Chad
You're talking about a factor of what? 100,000?
20:11 Patrick
I mean, it's, uh, cities of 25,000 have a total property value on the ground of $2 to $3 billion, you know, typically. Could be more than that, depending on what's there, but $600 million of additional value-
20:26 Chad
It's a factor of 3,000.
20:28 Patrick
It's a... Yeah, correct.
20:28 Chad
I was a little bit exaggerated there, but-
20:30 Patrick
A little exaggerated. I mean, you, you had to pull out the calculator there for a minute. But really? You didn't- you know, your board goes in, and they're like, "Oh, man, we've got all this extra value that's there." Nobody was like, "Where did that value come from?"
20:42 Chad
Yeah, it's, it's curious to me how you could go through... Well, I mean, I, I can't say that this has happened in Utah. I have no experience in Utah as far as the appraisal districts go and how that stuff works, but, um, it's not surprising to me that that kind of thing would happen. I would not be surprised to hear that it happened in places across Texas, but it's crazy to think that no one caught it. Like, uh, what are your QA, QC processes? Are you not mapping out to see where the changes were and making sure that they're... Like-
21:10 Patrick
Uh, I mean, just a simple variance analysis would've, would've found that.
21:12 Chad
A billion dollars worth of extra value?
21:14 Patrick
Correct.
21:15 Chad
No matter how fast your, your region is growing, um, an extra billion dollars is crazy. I mean, think, how much does a Walmart, you know, uh, store valued at?
21:27 Patrick
Oh, I mean, 14, 15 million.
21:29 Chad
Right. You have, like, a large multifamily apartment complex might be 50 million-
21:35 Patrick
50 to 60 million
21:36 Chad
... maybe 100 million if it's huge.
21:37 Patrick
I mean, a billion-dollar project, I mean, let's just like in, in, in terms of size and scale, right? Billion-dollar project, Southlake Town Center, everything that's been built there, every house, every brownstone, every store, every last bit of that development, probably right at a billion dollars.
21:54 Chad
It's a huge development.
21:54 Patrick
That is an enormous development. Yeah, I mean, that, that's, that's the scale of what we're talking about. I mean, it's just, it's wild to think that that doesn't get caught before somebody lowers the tax rate because of the new revenue.
22:09 Chad
Yeah, so yeah, so the, the issue then is, "Oh, we have all this extra revenue, so now we can either lower the tax rate and still capture some of the growth, or we'll keep the tax rate the same, but now we have s- $6 million worth of..." Yeah.
22:18 Patrick
Well, and everybody jumps at that point for new parks and new libraries.
22:21 Chad
Yeah, we got all kinds of new stuff.
22:22 Patrick
Yeah.
22:22 Chad
So now you have to lay off people because... or raise taxes to account-
22:25 Patrick
Correct
22:26 Chad
... for it. But, um, yeah, it's... I think part of the problem, especially with property taxes, um, when there, when there's a central appraisal, um, then the cities just get the numbers, is that there's not a lot of transparency. Because there's so much data, there's so many properties they have to look through to verify, you know, what the values of each property actually went from and went to.
22:48 Patrick
Mm-hmm.
22:48 Chad
Um, so cities get that number, and they just... You have to trust that it's correct.
22:53 Patrick
Well, I mean, having the trust that that number is correct and actually trusting it are two different things. I mean, we, uh, multiple times I've known friends in cities that have challenged the number that they get from the appraisal district, and that number changes quite often. It's not as set in stone as people really think it is. But I just have to go back to the armchair quarterback for a minute. Understand, I, I am taking the position of an armchair quarterback here. The assessor-
23:18 Chad
While you're sitting in an armchair.
23:19 Patrick
As I'm... While I'm sitting in an armchair. The assessor says, "Hey, this is now a billion-dollar property," okay? There's now $600 million of new, more net- there's... Sorry, there's a billion-
23:32 Chad
Yeah
23:32 Patrick
... there's like 900 and something million dollars-
23:33 Chad
It, it went from 300,000 to 987 million.
23:36 Patrick
Yeah, sorry about that. I mean, there, there's a, a billion dollars of new revenue that comes on the books. That assessor gives that new revenue to the city or the governmental entity that's there, right? You're gonna have a city or a county manager, you're gonna have a budget official, you're gonna have a finance officer, you're gonna have all these different people that look at this, and at some point, not one of them goes, "Wow, I don't think we had a billion dollar a year."
23:58 Chad
Well, so take a city of, uh, a city located here in the Metroplex, population about 30,000.
24:06 Patrick
Mm-hmm.
24:07 Chad
The property values on the ground are 2 billion?
24:11 Patrick
Correct.
24:11 Chad
Right, so-
24:12 Patrick
2 to 2.5 billion.
24:13 Chad
I mean, even in a city like, you know, a major city with hundreds of thousands of people-
24:18 Patrick
Correct
24:18 Chad
... a billion dollar increase in property values is gonna be noticeable.
24:22 Patrick
I mean, a billion dollar increase in values for a single development would be w- w- would be a-... would be the largest development being done in a major city, right? I mean, uh, I think some of the largest developments I've ever seen done are, like, the Facebook data center was right over a billion dollars originally. It's actually a lot more than that now, and that was in North Fort Worth, right? That's, that's one of the largest deals I've seen. Uh, I mean, really, that's how big that deal is. Like, it's national news. Yeah, and it-
24:49 Chad
It doesn't fly under the radar.
24:50 Patrick
It doesn't fly under the radar, and all of a sudden, your residential number ticks up a billion dollars-
24:55 Chad
Yeah
24:55 Patrick
... and you're like-
24:56 Chad
That's true
24:56 Patrick
... "We built a billion dollars in houses this year?"
24:58 Chad
That's true because they do separate, at least here in Texas, um, maybe they do it differently in Utah, but in Texas, it's sep-
25:02 Patrick
They do a lot of things differently in Utah.
25:05 Chad
It's-
25:05 Patrick
That's correct.
25:06 Chad
Uh, your, your, your values are at least separated by residential, commercial. Um, in Texas-
25:12 Patrick
Yeah
25:12 Chad
... there's mineral values and things like that.
25:14 Patrick
Correct, yeah.
25:14 Chad
But yeah, so you'd see the residential tick up by-
25:17 Patrick
Y- you would
25:17 Chad
... a massive amount.
25:18 Patrick
And you'd have to ask yourself the question, "Did I have that many new home building permits?"
25:23 Chad
I think that we underestimate the value of just, uh, quick gut checks on certain things. Like-
25:29 Patrick
Okay
25:29 Chad
... so, uh, when I was, uh, when I was building budgets and, like, managing the budget process for the city of Weatherford, um, that was always my, my big concern is, did I miss something big? Like, did I just kind of gloss over something early in the process and then forget about it later? So, uh, you know, we built a bunch of spreadsheets that would just compare different things and make sure that we could do just a quick gut check every, you know, every time we got to a milestone.
25:54 Patrick
Correct, yeah.
25:54 Chad
Here's just a gut check. Oh, you know, sales tax didn't accidentally increase by 30%. You know, it's a reasonable amount. Um, and of course, every time that you're updating your, uh... So your budget, when you build it for the following year, is based on the previous year's budget, but your projections are actually based on what you expect in the current year. So if you're looking at, uh, you may have budgeted 10 million for sales tax in the current year-
26:16 Patrick
Mm-hmm
26:17 Chad
... um, but that number is irrelevant when you're trying to prepare for next year's budget because you're looking at growth off of what you actually got, right?
26:24 Patrick
Correct, yes.
26:25 Chad
So halfway through the year, when you're starting your budget process, you're projecting 10.5 million in revenue, so that's your baseline for projections for the following year, not the 10 million in the budget, right? Everything is, needs to be based on actuals-
26:37 Patrick
Correct
26:38 Chad
... when you're projecting things like that. Um, so, so sometimes you could have, if you had, like, high growth in the current year, you could have a large percent growth change, and this always kind of scared me, 'cause I'd be like: Why am, why am I showing, you know, a 12% growth here, uh, over budget? And it's because the actual growth that we've seen in the interim has been higher than we projected. So the actual growth from actuals is maybe 6%. Um, but just the value of having spot checks and automated checking, um, I've come to really appreciate the value of this as I've done more software development. Because, uh, we do- we have an automated test suite-
27:12 Patrick
Mm-hmm
27:12 Chad
... and whenever I make a change in our software, we can run this suite of tests. It's, like, 150 different tests. It's testing almost every single line of code, and if I make a tweak here, and my tests break because I broke something that, you know, like, downstream, then I catch it. Um, it's very difficult, obviously, to build... Like, just the concept of, of automated testing in, you know, budget development or-
27:38 Patrick
As in-
27:38 Chad
... property appraisal-
27:38 Patrick
... property tax, yeah
27:38 Chad
... it's, like, difficult, but just the, the value of just giving a quick rundown.
27:43 Patrick
But that's just... Well, yeah, correct.
27:44 Chad
Do I have, like, an outlier here? Like, what are my top 10 taxpayers, right? And if one of them is a billion dollars, that's probably an issue-
27:51 Patrick
Yeah
27:51 Chad
... especially if it's a residential property.
27:52 Patrick
Correct, but I, I, I wanna, I wanna talk about a philosophical thing here when it comes to cities, right? 'Cause we see this a lot. When things are good, you don't ask a lot of questions. When things are bad, you ask every question, right? And, uh, there's probably a philosopher out there who said that, but if not, that's a, that's a Waller Cleese.
28:09 Chad
Well-
28:09 Patrick
There you go
28:09 Chad
... we see that too, right? Because-
28:10 Patrick
Correct
28:11 Chad
... when, when your sales tax is growing quickly-
28:14 Patrick
Correct
28:15 Chad
... uh, you're not... Those are sometimes more difficult sells for us.
28:19 Patrick
Well, I mean, I mean, if-
28:20 Chad
But if your sales tax drops-
28:21 Patrick
Yeah
28:21 Chad
... 20% over the course of six months, then you wanna know why, and so those are, those tend to be easier sells.
28:25 Patrick
We're, we're always, we're always very kind to clients who have, like, 20% growth in their sales tax and don't want... You know, they're just like, "Hey, we just don't need y'all's services." And we're like, "Hey, fully understand, right?" But they're also the first people to call us when their sales tax is down 10%, and it's like, hey, all of a sudden... You know, we, we've, we have a lot of cities that have become clients in the downside, the downswing. I, I don't wanna go as far to say that our, our business would be better in a downturn, but it is one of those things where cities don't tend to care why things are good, but they always care why things are bad.
28:58 Chad
Yeah, it's, um... So I, uh, we both came into the field during the '08 recession.
29:04 Patrick
Oh, yeah. We were the, like, the worst MPAs coming out in the world-
29:06 Chad
Jobs
29:06 Patrick
... 'cause jobs were hard to find.
29:07 Chad
So I was in the budget office-
29:08 Patrick
Yeah
29:09 Chad
... which is countercyclical.
29:10 Patrick
That's correct, yes.
29:11 Chad
Or at, or at least to the point of, uh, I'm probably not gonna get laid off, because we need people to evaluate where things are headed and help us make decisions about how to, uh, accommodate those changes.
29:23 Patrick
I, I was in community services and special projects, so I found another job.
29:28 Chad
Yeah, so okay, I think we may have exhausted that topic, so on to the fun portion-
29:33 Patrick
Yes
29:34 Chad
... of the talk. Um, I recently rewatched one of my favorite movies. You probably haven't seen it. Uh, it's, I think, 2001, Jake Gyllenhaal classic called Donnie Darko.
29:48 Patrick
I have not watched this movie.
29:49 Chad
Okay.
29:50 Patrick
No.
29:50 Chad
It's not that important-
29:51 Patrick
Okay
29:51 Chad
... for the conversation, but it got me thinking, 'cause Donnie Darko is about time travel-
29:54 Patrick
Okay
29:54 Chad
... to some degree, at least. It's about a lot of things, but time travel is an element, and it got me thinking about all the different movies that incorporate time travel and the different rules. They, like, they all have different rules about how time travel works.
30:07 Patrick
Okay.
30:07 Chad
Okay, so, like, if you look at Back to the Future, um, they can... Going back in time can affect the past, right?
30:15 Patrick
Uh-huh.
30:15 Chad
And, in fact, I think that they actually describe what happens is, like, an alternate reality, like a, like a-
30:20 Patrick
Going back in time can affect the past, or going back in time can affect the future?
30:24 Chad
Well, both. Like, if-
30:26 Patrick
How does it affect the past?
30:27 Chad
Okay, so if you go... Well, if you're in 1985, and you go back to 1955-... and you, and something happens, like you end up, your, your mom ends up with a crush on you instead of your dad.
30:40 Patrick
Correct.
30:40 Chad
Okay?
30:40 Patrick
And you're wearing Calvin Klein underwear.
30:42 Chad
Yeah, so that affects your future, but it also affects all the past that led up to your future.
30:46 Patrick
Okay.
30:47 Chad
Right, so it can kind of affect the past techno- Not... It, it wouldn't affect things before 1955, but it would affect the past relative to-
30:53 Patrick
But if you're traveling to the- if you're traveling to the past, you would be affecting the president what- present, which then affects the future.
30:59 Chad
Yes.
30:59 Patrick
Okay.
30:59 Chad
Yeah.
30:59 Patrick
So just wanna point that out for-
31:01 Chad
So but at, whenever you make a- w- whenever something monumental happens, it creates like a, an, uh, like an arm, like a separate timeline.
31:09 Patrick
Okay. Okay.
31:10 Chad
Okay?
31:10 Patrick
Yeah.
31:11 Chad
Um, and so you can see that because, you know, like, the picture of his siblings, they start to disappear and all this kind of stuff, and they, they actually go back after... In the second one, they go back, uh, or they go to the future, and, uh, they get the almanac-
31:25 Patrick
Okay
31:25 Chad
... which then old Biff takes back to 1955 to give to young Biff, and then when they go to 1985, it's totally different.
31:32 Patrick
Okay.
31:33 Chad
Right. But the idea is that you can't go back in the past and not affect the future. Contrast that to, like, Bill and Ted.
31:39 Patrick
Okay.
31:39 Chad
Okay. So they're, they're going all back in time and picking up historical figures, then taking them to the future. And, uh, although they... It's not clear that they tell everyone what happened to them. Like, they bring Abraham Lincoln, and I don't think they tell him, like, "Hey, don't go to Ford's Theatre." But, uh, there is a scene where Napoleon is explaining his, uh, his plan for Waterloo, uh, because he really liked the Waterloo, you know, uh, water park. And so he's on the stage, and he's explaining, and Bill, I think Bill says, "I don't think that's gonna work," or maybe it's Ted. But, like, so it's kind of implied that they have, at least in some cases, given hints about what those historical figures' future is gonna be like, but it has no bearing, obviously-
32:24 Patrick
Okay
32:24 Chad
... on the future as the, you know, as they go back to their original time and then move forward. Um, but it's not consistent because when Rufus comes back into the past to alter the future, right? So it's w- maybe it's just 'cause it's a comedy, and it's supposed to be mindless, but it's always bugged me that there's no con- coherent worldview of how time travel works.
32:49 Patrick
Okay.
32:49 Chad
But my favorite is Donnie Darko, um, because it is a... The way time travel works in Donnie Darko is not so much that you can go back and forth through time on your own. Um, there have to be, like, a confluence of events and circumstances that allow you to do it, but you can actually see what's gonna happen before, uh, it happens. So, like, you can see this energy moving from your body that's telling you, like, where you're about to go, and then you kind of, like, follow it, or you... The question is, uh, do I have to follow it, or can I change? Like, if I can see the future, do I have to follow it, or can I change? And we kind of talked about this earlier off the air-
33:30 Patrick
Correct
33:30 Chad
... about, about Minority Report, but-
33:32 Patrick
Yes. So I don't watch a lot of time travel movies. So the only example I have is, I can't remember if it... I think it was a Sandra Bullock movie. I don't remember the name of the movie.
33:44 Chad
The Lake House?
33:45 Patrick
The Lake House.
33:46 Chad
Yeah.
33:46 Patrick
Yeah.
33:46 Chad
Also, Keanu Reeves.
33:47 Patrick
Kea- Keanu Reeves is in that movie. He's an architect, right?
33:50 Chad
Mm-hmm.
33:51 Patrick
Okay, and, um, they write letters to each other, and they put it in a mailbox, and this movie always confused me because somehow they were able to meet and know each other without actual time travel.
34:05 Chad
Okay, so the way that this worked-
34:07 Patrick
Yes
34:07 Chad
... if I remember correctly, 'cause I think I've seen it once-
34:09 Patrick
Okay
34:09 Chad
... and that was years ago. Um-
34:11 Patrick
I thought it was a very nice movie.
34:12 Chad
So Keanu Reeves designed this lake house.
34:15 Patrick
That's correct.
34:16 Chad
And Sandra Bullock lived in the lake house.
34:20 Patrick
He designed it. She lived in it.
34:23 Chad
At a later... In, in the future.
34:24 Patrick
In the future.
34:25 Chad
Yes.
34:25 Patrick
Correct.
34:26 Chad
But they would write letters to each other and put them in the mailbox, and then when they put them in there, it would transport into the future or the past, and the other person could read it, and they built this relationship over this time travel letter-writing.
34:37 Patrick
Correct.
34:38 Chad
Uh, and then at the end of the movie, their stories converge, and, um, s- I, I don't remember... I, I assume that Sandra Bullock was in the future because-
34:48 Patrick
So, so at some-
34:48 Chad
... she lived in the house that he built.
34:49 Patrick
Yeah, at some point, I can't remember exactly what happened, but at some point, Sandra Bullock happens to be at his architecture firm, and she sees a, a, a architectural rendering of the lake house on the wall, and she figures out that it's him, if I remember right.
35:06 Chad
Yeah, and it ends tragically.
35:09 Patrick
It doesn't tragically, right? Yeah.
35:11 Chad
One of them gets hit by a bus, I think, when they're trying to meet, and then the other one... You'll have to look it up real quick, but then the other one goes to see what happened at the bus accident, and they find that it's that person. It's very tragic.
35:23 Patrick
But doesn't he go back to find her before the accident happens?
35:27 Chad
I don't know. I have a, I have a feeling that this is gonna be terrible listening for anyone who's still following, but, uh-
35:34 Patrick
It was, it was a 2006 American romantic drama film, so-
35:39 Chad
A rom-dram?
35:40 Patrick
A rom-dram, yeah, The Lake House. Um, I, I mean, I don't even know how to look this up.
35:45 Chad
Wikipedia? IMDb. Do you know how to use the internet?
35:50 Patrick
I, I am, but I, I don't, I don't look up... I'm not a movie buff like you are, although I've watched a lot more movies lately than you have, which is weird.
35:59 Chad
We have- we used to go to the movies, like, every week, and then we just haven't done it as frequently with- once the kids got here, once they showed up on the doorstep.
36:07 Patrick
Once they showed up on the doorstep. Nice.
36:11 Chad
Okay. So here's what happens. So their timelines are separated by a couple of years, right?
36:18 Patrick
Correct, yes.
36:19 Chad
So he's writing while he's living in the house in the past.
36:22 Patrick
That's correct.
36:23 Chad
And she living, she lives in the house in the present.
36:25 Patrick
Correct.
36:26 Chad
Okay? So as time progresses, they decide to meet. He ends up not being there. She gets dis- disappointed, and then one day, like Valentine's Day, she goes back to the house, and they get back in touch, and they decide, um-... or I don't know, something like that. But she's going to an architect to look at plans for a new house that she wants to buy, and that's where she sees the picture of the lake house.
36:49 Patrick
Yes.
36:49 Chad
Okay? And, um, it's her brother is the architect that she's talking- his brother is the architect she's talking to, and, and Keanu Reeves, she finds out, died in this traffic accident, getting hit by a bus or something, um, on this specific day. So she goes back to the lake house and writes him a letter and says, "You know, you're- this is... Like, don't do this. This is gonna be bad." And, uh, it, anyway, it ends up where basically, like, there's nothing they could do. It's like fate.
37:16 Patrick
Okay.
37:16 Chad
And he ends up trying to go meet her and gets hit by the bus. So it's almost like a self-fulfilling prophecy kind of time travel.
37:22 Patrick
Okay. Interesting. We went way off track with that one.
37:28 Chad
That's what you get for-
37:28 Patrick
But that's, like, the only time travel movie besides, like, Back to the Future that I've really ever watched.
37:33 Chad
What about-
37:34 Patrick
I didn't watch Bill & Ted.
37:35 Chad
You've never seen Bill & Ted?
37:36 Patrick
I've s- I mean, I've watched it, but, like, I don't remember it.
37:40 Chad
Really?
37:41 Patrick
Yeah, the new one's just came out or is about to come out?
37:43 Chad
It's about to come out. You should probably watch the first one.
37:45 Patrick
Okay.
37:45 Chad
I don't... Uh, the second one, I never really cared about-
37:47 Patrick
Okay
37:47 Chad
... but the first one was entertaining.
37:49 Patrick
It was good?
37:50 Chad
Yeah.
37:51 Patrick
Eh, I-
37:51 Chad
I mean, I was, like, seven-
37:52 Patrick
Okay
37:52 Chad
... when it came out, so, eh, it's tough to remember. I haven't seen it since the early '90s.
37:57 Patrick
Like, I watched Ford v Ferrari, and you said you didn't, you... That's not your kind of movie.
38:02 Chad
I don't even know what it... What is it about?
38:03 Patrick
It's about Ford competing in the, is it Le Mans?
38:07 Chad
Le Mans?
38:08 Patrick
Le Mans.
38:09 Chad
24-hour race?
38:10 Patrick
Yeah, the 24-hour race. So it's about Ford going over there basically, like, right after World War II, you know, '50s, I believe.
38:17 Chad
So which one is Ford, Matt Damon or Christian Bale?
38:20 Patrick
So Matt Damon is, Matt, Matt Damon plays Richard Shelby, which-
38:23 Chad
Okay
38:24 Patrick
... any car person knows Richard Shelby. He's, like, one of the greatest engine designers of all time, so he is one of the greatest engine... I'm a Ford guy, so I care about that one. So, but yeah, um, I mean, that, that's a good movie. I watched Midway. I thought Midway was a good movie, so.
38:39 Chad
I haven't seen that one. I saw 1917.
38:41 Patrick
I haven't... Was that good?
38:42 Chad
It was good.
38:43 Patrick
Okay.
38:44 Chad
Yeah.
38:44 Patrick
Okay.
38:44 Chad
So, like, the filmography was interesting.
38:46 Patrick
Okay.
38:46 Chad
Um, I know it's not the first single-shot movie, but I think it's the first one-
38:52 Patrick
So they did a-
38:52 Chad
... that I, it's, it's-
38:53 Patrick
They did a war movie as a single-shot movie?
38:54 Chad
It's shot as a single take.
38:56 Patrick
The first one where is The Blair Witch Project?
38:59 Chad
That, I don't think that was a single shot because it was shot in the first person. That was the first, like, handycam or, like, self, self-documented film.
39:06 Patrick
Okay.
39:06 Chad
But they had multiple cuts.
39:09 Patrick
Okay.
39:09 Chad
Right?
39:09 Patrick
Yeah.
39:10 Chad
'Cause it's taking place over time, and it cuts between, like, "Oh, something just happened, so I turned the camera back on."
39:15 Patrick
What was the movie that won all the awards, like, two years ago that was terrible, but it won awards because-
39:18 Chad
Bird... Is it, um, Bird Land?
39:20 Patrick
Bird something. It was-
39:21 Chad
It's the Michael Keaton movie, right?
39:22 Patrick
Oh, I thought it was horrible.
39:24 Chad
Birdman.
39:24 Patrick
Birdman, yeah. I, I really did not like that movie.
39:28 Chad
But that was a single-shot movie, right?
39:29 Patrick
It was.
39:30 Chad
Yeah, I did see that.
39:30 Patrick
It was like the one that made single shot famous, right?
39:32 Chad
I know there was a long single-shot, uh, opening to Gravity, another Sandra Bullock movie.
39:38 Patrick
Uh-huh.
39:39 Chad
Um, but I just find that... So the difference in 1917 is that it was, um, obviously based in World War I, so there's like, they're running through trenches, and there's just all kinds of extras, and, um, they have to manipulate where-
39:52 Patrick
That's gotta be incredibly difficult to do a single-shot movie as a war movie.
39:56 Chad
Yeah, and l- like, especially the beginning, like, the first 10 minutes, they're just running through trenches, talking to people and getting orders and doing things, and so, like, having to maintain dialogue, and they have to, like, shift their, you know, where they are. Like, "I'm in front now," and, like, looking back to talk to you, and then they switch, and they go around corners, and it's... Yeah, I, I can't imagine how difficult... Like, you can see where, you know, some of the cuts were made, um, especially, like, as they go into a, a room that's inside, and there's like, you know, it's dark for a second.
40:28 Patrick
Yeah, correct. Yeah.
40:28 Chad
So, um... And there's one scene where, or there's one part where, um, there's, like, a break in the action because there's a s- spell of unconsciousness, right? So-
40:37 Patrick
Correct
40:38 Chad
... it's not technically a single shot, but essentially it is. But it, I thought it was really interesting from a, from a filmmaking stand- like, a storytelling standpoint, to tell that movie because it really made you feel like you had no idea, like, you didn't have any context about what was happening, which was basically what these characters were feeling. Like, they didn't have... They're just kind of going out in the middle of this war on their own.
41:00 Patrick
Correct.
41:00 Chad
So like, they don't really know where anything else is and what else is going on, so they're just as uninformed as you are. Whereas in a normal movie, you'd, like, you'd have more... You'd have a wider shot. You could see things that are happening. You could kind of tell if, you know, someone was behind the, the corner over there, so.
41:15 Patrick
Well, I mean, like, at the end of the day, um, your movie taste is very different than mine. It just is.