Window function 😊. CTE with window function 😮. Hierarchical recursive CTE 🤯. Recommended all the way. Can’t wait to see the rest of the videos!M Wildan Zulfikar
Shorten dev cycles with branching and zero-downtime schema migrations.
Okay welcome back uh to database school I'm your host Aaron Francis and I'm here today uh with my friend Jesse Hanley um Jesse runs a an email service that I actually use and I pay full Freight for so even though we're friends you got you gotta you got to pay your friends and so I use Bento uh love it we use it for all of our courses we use it for Aaronfrancis.com we use it for everything so Jesse's running a pretty a pretty big operation over there and he does it very in a very lean way so I'm excited to talk to him today Jesse do you want to introduce yourself yeah sure I'm Jesse uh Aaron is a customer of mine which I appreciate very much um but yeah I run Bento which is an email service provider we predominantly send marketing email but recently have got more into transactional email password resets and all that kind of jazz um and I kind of run it out of my home in Japan and have for the past four years in Japan so yeah yeah it's a fun very calm business to run so um I very much kind of enjoy running it here this blows my mind that you say that email and being an ESP is a calm business to run how first of all how many is it it's just you and you have some customer support is that right to cover some of time zones or yeah so I'm I'm the primary developer and then I've got Anya in support she covers kind of the eastern and the US time zones when I'm sleeping um and so if something goes down then I get a call wake up and I'm on call and all that kind of jazz we also have uh contractors for various different parts of the operations so for example at the moment I've got fast the fast Ruby team they're working on upgrading our you know main monolith uh app and all that kind of stuff get and we'll kind of go into why they're working on that at the moment because that kind of ties into some of the other stuff uh database related stuff MH um I've got like deliverability Consultants that I work with I just kind of bring in experts and so the way that I've tried to build the team is less kind of fulltime on the payroll stuff and more kind of experts contractors that kind of either work consistently or they kind of come in for very specific jobs coming in and then bounce out um and i' I've always liked working that way it's kind of really fun yeah and it's it's shocking to me because you know you run you run support out of a Discord that is very active and um you are very active in there and I'm like when does Jesse one when does he sleep and two when does he develop because I know he lives in Japan he's in there talking to customers like me who can't who can't figure out how to do something you're like hey use the button that's right in front of your face and I'm like oh yeah you should put a button there so how do you how do you like manage your time that you're always in this Discord or responding to my DMs um uh well so so like Discord Discord is like there a little bit of a tangent discord's really nice for support because I I I've got it on my phone I can quickly message people respond really fast um luckily like the software is not that buggy touchwood so like often it's it's a user thing like they've accidentally deleted their DNS records in a migration so they just got to add them back again and boom their open rates back up um so 90% of the tickets are like either user errors or they're a little bit confused or they just need to like learn something and so Discord is a nice medium and also people are like less trying to work out like the PC way to say this people are like way nicer on Discord than they are on email on email people are like awful they're like super rude super blunt Dear Sir dear ma'am who whom This concerns F you I've got an issue whereas on Discord because there's hundreds of people observing them they're super chill right and I like like that I like that Dynamic of it so from like a support perspective it's very weird for customers to to Discord but when they see like a Founder reply they're like well this is awesome I'm getting like a Founder response they get a response very very fast which is great um but also the exchange is a lot friendlier than elsewhere so I I kind of estimate that if I had to do email support like I only get like six emails a day eight emails a day like on on email support which is really really low um but that's actually been consistent for the last year despite growing almost double in the last 12 months so that's been ing to me and I think like if I didn't have Discord maybe two to three full-time support people probably responding to tickets and they would have to be more skilled and all that kind of stuff I also think people a lot um more chill in terms of response times on Discord like they're okay waiting a little bit if I'm sleeping um because on your response straight away says Jesse will get back to you and like just that quick response is that that's totally cool so yeah it's a different way of running the business but I actually think it it's Cal it's weird at the start but it's it's a calm way to do it yeah it is it is weird at the start and as a customer coming into a Discord and seeing all the activity made me feel like whoa there's a lot going on here so to hear you say it is calmer I guess the activity was going to happen either way like you were either going to get emails or people were going to ask in Discord um but the the observation effect is very interesting that people are better behaved when they're being observed that is very clever hey we'll get back to the interview in just one second but since you're here you like databases you probably like Postgres let me tell you about Mastering Postgres this is a course that I am teaching coming out October 15th this is a way that you can master the most important part of your stack your database so go to masteringpostgres.com and sign up to hear when we release it okay back to the interview yeah and uh we've had instances in the past this hasn't happened like in a while where um European customers tend to be a little bit more like blunt especially like Dutch customers they tend to be very blunt with their feedback and we've had instances where people are being a little bit too blunt with their feedback and then almost like I don't know blood cells on like a virus or something they all just like come in kind of defend um and that's really nice because sometimes if you get an email that's like super blunt you're like oh I'm a I'm a bad software maker I'm going to shut this business down but in the community thing it's it's just a little bit more chill um and I think in addition Discord has been like a another benefit because like if there is an issue like I I accidentally shipped like a bad conditional and a button that wasn't supposed to be hidden was hidden this happened yesterday like pushed up some UI changes right and people come to the Discord so fast with where's that button and I can quickly respond all right it shipped 15 minutes is fixed so anytime that there's an issue it's like such like a yeah I just get this like live pulse through the Discord that maybe QA or you know a test can't really pick up um and that's been really good or maybe like a button's missing people happen to come in and be like where this thing um in Discord but they're never going to email me that cuz that's so much effort but on Discord they're happy to it so there there's a whole bunch of weird kind of benefits to running a business like that yeah okay that's interesting because I I had kind of always wondered why Discord but this is all this is all making sense now and so tell us a little more you are uh fully bootstrapped and if you have the fast Ruby team it's all Ruby on Rails is that right mhm yep so and how long how long have you been doing this uh I've been doing Bento like six years the first like four years I was running uh my agency full-time and then I exited that in covid and then I went fulltime onto Bento and then kind of like really found my footing for the most part I was like running an agency a marketing agency and learning to code on the side and the way that I kind of improved my skills was I brought on really smart people like Andrew Culver I brought them in I learned from them and then would take over more of the code base as I kind of got them to build stuff almost for me and then I would go okay cool like that's how you're supposed to do it properly like that's how a pro does it and and every kind of core feature that I've built I've done it that way I found an expert built with them learned from them taken over the code base and kind of that's been the process with Bento so yeah and the first four years was really just trying to like find my foot and it took a really long time like there was lots of features that we launched that no one used those lots of products I was trying to like push on people that they never used um and then after I went fulltime I kind of really narrowed in on the email marketing side of thing I was like well like everyone like uses these features and they like using these features I'm just going to keep pulling on that thread and after I kept pulling on that thread like actually start to get some traction and then the last two years has just been no we only send emails and the more we we we we leaned into that um the more people signed up and the more people were like I need to send emails I'm going to try Bento and um yeah yeah amazing and so where did you find as as you're like honing in your message where are you finding that people are hearing about you like how it is a very uh it is a very crowded Marketplace in terms of email which is a great thing because everybody needs to send emails so it seems like you know Justin Jackson always talks about catching a wave and emails still a wave but where are you like carving out your Niche amongst uh the other competitors yeah the way that I kind of like to explain it is bent for the most part is kind of like a local coffee shop of email marketing providers right so where yeah you're right you've got like mail champ active campaign they're like the Starbucks or like the tul coffee of of chains um but just like a local coffee shop people do tell others we get customers from Twitter or referral or you say it or like someone tweets about it or some indie hacker in barley mentions it or someone searches for like a learn Das email marketing solution and come across that we integrate with them um there's a lot of stuff like that which it's just kind of breadcrumbs that have been left on the web and people discover these breadcrumbs stumble and then we become a category decision so we become a decision out of many of the providers um and then our job from then is to be like persistent so we've got retargeting ads so people keep seeing us um and so that when they do make the decision to switch they do decide to choose us um and then often it's because they've done a demo and they're like okay okay cool I've got big company big company local coffee shop and I can actually speak to Jesse I've done a demo with him he's active in the Discord uh I can see that if someone has a very specific email not being delivered he's given the exact reason why he's not gaslighting me I'm going to go with that dude um and that's how we win business and then from like the bootstap perspective it's like well email is profitable this like you know our customers we price we don't have free plans we charge for everything I charge you charge everyone and and because of that it's like a healthy profitable business um and so I'm not too concerning growing a lot faster um and we've got you know this morning I like banned five or six signups in the past couple of days I'm just not interested in serving them and so we don't have massive amount of signups and so we can kind of keep a float that way if that makes sense so yeah YouTube videos it's just all over the place yeah and having been around for six years with the good track record of supporting your people I imagine the word of mouth starts to compound a lot more um a lot later in the game now so you've been you know rolling this snowball for six years and now people are like hey let's talk about Jesse because I've been I've been here for a long time yeah there was this um I won't mention his name but there was this email copywriter I think I like sent you in in DMS go go check him out um and I followed him from like 2010 to 2015 like reading all of his stuff right and and then he heard through I think Brennan Dunn like referred him somehow and it's just like everything's connected in this weird sphere where it's like this guy that I followed over a decade ago spoke to one of my friends and then now this guy that I super respected is considering moving to us next month and it's like whoa how like how'd that happen but you just you just don't know and I think the the takeaway for me is always like you just got to stay in the game a really long time and like the key is just surviving and if you survive stuff works out like and as long as yeah you're not St survival I I think it all kind of pans out okay yeah and it sounds like you've optimized for survival with the no free tier everybody gets charged a fair amount but everybody gets charged and you're going to ban signups that would I assume end up being super problematic for your reputation like your sending reputation or your other customers or something like that and so you've put in some guard rails uh with an eye towards survival it sounds like yeah yeah last night I banned someone in Austria are uh cuz they kind of hurt our IP reputation uh on one mail stream and it turned out it was uh one of my friend's girlfriends that signed up for a free trial and so I had a lively debate with him last night about that approval I just thought you was some some Austrian customer and I was like all right bye J Tri and he he was not happy but yeah we got a very strict bar and once she cleans up her list she'll be back in um perfect well good luck to her and good luck to you and your friend's relationship [Laughter] he's a good CER okay so I want to talk Tech choices a little bit um so we've talked about rails you had Andrew Culver so I assume this is uh either originally or or still a bullet train application which is bullet train install number one you're the first bullet train origin origin origin origin customer number one origin customer number one so um we like I did not know that so so bullet train was built with Bento at the same time so they really that that's kind of how it started right so the the the origin of bullet train Bento was kind of the base project um that was being dog food edible
like uh it's more than an application starter kit it's it's in the it's in the rails space uh its closest uh analogy would be an application starter kit but it's more it's more like a framework on top of a framework because there's lots of generators and Stu yeah there's lots of ongoing code generation and stuff like that um versus just like you get a starter kit and you eject and you keep going um so is it still like have you outgrown it is it still bullet train down there yeah we we we pretty severely forked like I kind of went off off the rails I went off the rails a a while ago um and so yeah no but also like different direction Andrew stayed I think from I got to go check out the repo but I'm pretty sure he stayed like relatively uh close to cor like stimulus and used all that kind of JavaScript stuff I wasn't so interested in that I I like react um I was I was react curious and then I started really liking it after I started playing with it a lot more um and so a lot of like front end stuff in Bento is like react sprinkles on top I use react components and then you know like our flows automated you know the diagram automation Builder that's all just in react so um yeah there's no way you're doing stimulus yeah yeah so the uh what he's talking about is there's a like automation engine kind of thing and it's like on a canvas and you can drag nodes in and connect them and it's it's extremely powerful and also extremely impossible to do with HTML alone so that makes that makes sense to me um okay so let's talk Postgres so your rails I gather you're still hosted on Heroku yep yeah y so we've got um so we've got uh two uh apps on Heroku we've got our kind of monolith app and then we've got our chat app um chat's like two-way conversations um they both live on Heroku uh and then we also have our own San servers so we use a couple of third party API providers but we also have our own uh MTA Network and that is actually physically what delivers the emails to inboxes so when Aaron francis.com sends an email it goes to our MTA and we deliver it not srid postmark all that kind of stuff so really um yeah for like for the last year all of your personal emails have gone through like bento's network not through any third party uh provider um and that's know let's take a little yeah let's take a little pause from postest you're running your own mail servers isn't that like hard mode yeah it's extreme dude so why are you running your own mail servers I don't know because
um I don't know I thought it was a good idea it's been it's been very traumatic um it's been it's been it's been quite a traumatic experience but um it's been quite enlightening um Let me let me I just laughed so hard I almost cried uh so uh yeah so it's been it's been it's been an experience so I decided to do it because obviously as we scale um sending more emails cost more money right yeah um and also what happens if like one of the providers I use go down I I need I need redundancy in the system or let's say like an automated system this probably won't happen because like I've got Enterprise accounts everywhere but let's say like an automated system uh bans my account I need to send I I can't not sent um that's not my problem that's your problem so that's my problem yeah which sounds terrifying I needed a way to have like resilien resiliency in the in the network um and so having my own service was an exploratory project for me um and so I did explore that I won't go into too much detail on like how I do it um but the end result is we've got servers up we have multiple IP addresses uh we send on them and it's it's very very different it's more like you know like last night I have like an open ticket with Hotmail asking them to like mitigate some of my new IPS and stuff because you know you you got to do that or I got to do what is it preemptive mitigation and all this it's just it's just very very different um and so yeah yeah it's uh yeah so we' got our own email service I don't know say I first of all I envy you 0% no part of me envies what you do second of all um besides economics has it been better for any other reason better deliverability better control fewer issues or is it just like redundancy in economics uh it's redundancy and economics which are honestly frankly like two very big Parts those are big Parts like the it's it's a peace of mind thing right which we're probably going into when we talk about Postgres it's like everything that I'm trying to do is for peace of mind um and so yeah I I like that aspect of it um um just feels nice to have my own stuff I don't know it's just cool to have like my own sing servers you know that I control and I own that's all on Hetzner you know which is the the current Trend that's what everybody's talking about yeah and they're great and it's like quite empowering having my own box on Hetzner that sends emails and um all that kind of stuff and Hetzner honestly been a pretty good provider they opened up the ports which is something you need to do um to allow us to actually like send out and like they're pretty good when they deal with um like uh they get abuse requests and stuff like at one point a whole bunch of like Brazilian isps was spamming them with abuse things that they' like just I know just weird stuff right but hon was pretty good at actually like talking through support not to shut down our box and all that so they're a great provider I I I really I like them I like them a lot actually um but yeah it's quite impar once you get that box up and you're sending zero cost emails you're like that's different you know yeah got a little a little Taste of free emails I'm sure that was exciting this is very I I mean this is very on brand for the current rails Trend it's more fun to be competent host your own stuff and use Hetzner that's like that's big rails right now so you're right in line yeah yeah exactly okay so that stuff is living over on Hetzner wild decision to roll your own male servers but mad respect um talk to me about uh Heroku mhm yeah so so on on Heroku we have our like monolith app which is like a rails app um we're on we're working towards rail 7. something um and that is primarily a big Postgres instance just like a single Noe instance uh then we have multiple elastic instances supporting that and I'll go into like the reasons why we have lots and lots of elastic instances um and then I have like three Rus instances on there for a whole bunch of different things just separating concerns background jobs uh caching layer and Etc so yeah websockets all that kind of stuff and Bento chat is very very similar to like that stack as well and but it's its own stack it's got its own database and and everything yep yep yep yep yep y okay cool um I assume you've been on Heroku since the very beginning and just have continued to turn the dials as you've grown is that right yeah yeah yep and I've liked them for a few reasons like a kind of meta reason that I've liked it is with Heroku once you start spending a lot of money and you will spend a lot of money and you um you can sign Enterprise contracts now why that's beneficial to me is if there is an issue let's say someone's account gets hacked or let's say someone does uh let's say our spam API doesn't catch something and we start sending out a whole bunch of emails that shouldn't be sent out that that is a risk right
and but on the that it and a ticket gets raised to Heroku if I am on their general population they will shut me down most likely right until I respond to a ticket they'll just they'll shut traffic off or they'll they'll do something because I've got the is Enterprise flag that is much less likely to happen and so what I what the advice that I've gotten speaking to a lot of insiders that and actually just in general across any part of my stack is like you want to get to Enterprise as quickly as possible just so you have that is Enterprise flag um yeah same with like Cloudflare once um I had like a URL redirect abuse thing going on someone someone managed to find find that they could uh shove Us in between their redirect chain and use us that got to Cloudflare and Cloudflare was great about it they like it was a ticket they're like sort this out I did and it was all good but I I'm on an expensive plan there so I think they were less likely to shut my traffic down straight away so that that that that's why I've stayed predominantly with Heroku for the last um yeah since the beginning do you also get and if you're not allowed to talk about it that's fine do you also get cost savings like do you pay a bunch up front to reserve some stuff or you're laughing I don't know if that's good or bad yeah they they changed the they changed like the pricing Dynamics a little bit um and you now get to talk to a sales rep which you can discuss things with and you can potentially get discounts um I don't mind that I don't mind I don't mind the negotiation thing um what I don't like is uh did you see that tweet that I posted about it was like uh um you can put like a Reds production instance on a little USB drive or whatever it was it was that thing and then that I posted that and then I was like I wonder if Heroku have like changed their pricing at all in a decade and I went to web archive it's the same they r the same in a decade in a decade so 201 like one of their instances 200s a month right one gig of um uh RAM it's the same today um which is crazy to me and on that Postgres instances a decade I think like all instances are maybe like beefed up 50% or something a little bit more iops but they haven't updated their pricing sheet in 10 years uh which I'll tell you how that makes me feel a little bit later but um yeah it's it was quite eye opening I did that last week and I I kept like writing a tweet and then deleting it I was like so upset I was like I'm I'm just going to I'm just going to like let this one slide and I'll think about it for a little bit but yeah 10 years keep thinking about it because that's a good tweet yeah you know maybe take a little bit of the edge off a little bit of the the frustration but that's a good tweet you got to tweet that that's good yeah minding they've been great they've been great provide I just want to say something they've been great great Pro uh vendors and they they uptime and reliability since booting off all their free users is very good so I oh you actually saw you saw a difference there yeah yeah since um I think it's Bob Wes since Bob Wes took over um I think from like the infrastructure and like reliability perspective Heroku has been like phenomenal and great from the pricing and uh product side very slow but I think that's like the Benny off grasp on that on that inside the Borg now yeah that makes um okay so you're over on Heroku you must be storing unbelievable amounts of data because anytime someone clicks on anything in any email that I ever send you you write it down you're very diligent you write it down you put it in the database and I I'm a little guy so I imagine you've got people sending out hundreds of thousands of emails a month and every single click is coming through you to say nothing of all like the actual emails that you're sending so what what in the world how are you storing all of this stuff in one massive database um most of them are just in like basic basic tables an event table we got like an emails table um so very very simple kind of setups all the main pieces of data stored in there we've got like some Json B columns um just store kind of the more blobs which which then gets pushed to toast I would assume for the most part um which hasn't uh to date hasn't really been like I've seen I've read a lot of articles about like issues storing a lot of stuff to toast but we don't query a lot of that which is where elastic kind of comes in um so I haven't we use we use post kind of like as a a primary data store more so than anything else but not as our query store um yeah and some columns as well like our emails one they're very very tiny like it's not that much data because I store the HTML in S3 um and so I don't actually store that right so I I send the email storying S3 and then it's kind of like done right so there's not really um too much storage issues there um so yeah that that's they're very they're very lean Rose they're not too complex um so it's hasn't really been too issue yeah so this is interesting so you're storing potentially billions of of rows in here but you said you're piping a lot of it out to elastic to do yes the actual querying so walk me through that because that seems very clever so the way that I've kind of thought about Bento is because I've there's a couple things that I've always wanted to be a true of Bento I've always wanted to be light in Fast and I've always wanted it to be calm for me personally to run um and so a lot of the way that I've like architectured Bento is I'm doing a lot of like just select ID queries I'm like selecting the exact records I'm doing an association there indexes on that user. emails user do events right so that's that's always very very light in fast when I'm doing more broader stuff like a query over the whole tenant so over the site so Aaron Francis I want to find all of your users querying that in Postgres which like I'm sure is is quite possible um it's hard to get right at all scales just because like people have so many different types of uh pieces of data they got different custom Fields they've got just it's it's very hard to understand and map to everyone's world and create indexes for everything yeah so give I'll give you an example cuz like when I send out an email to uh when I send out I sent out an email maybe yesterday or the day before and I sent it to everyone that was on my list so that's already narrowing it down from many you know hundreds of th millions of subscribers to my little you know my little adorable list but my list where people have this tag list colon sqlite they don't have opt out colon sqlite and we didn't want to spam people or rather send too many emails so we also left out list colon postgress so it's my users with these two tags but not with this tag and that's a hard like that's a hard query that's a hard query to optimize so how are you like how are you thinking about that so that's a hard query to optimize 4 for post grass but that's not a hard query to optimize four for elastic that that is is like like bread and butter chips yeah it's it's bread and butter for elastic right um and so the the the pipeline is is like new data comes into our system the user gets updated we push that off to elastic search it gets indexed and then it's it's instantly queriable more or less right um and the reason I wanted to do that is because I just didn't want to run migrations or like let's say I had like um uh new user request a user goes hey I want to just find engaged uh people who engaged with my last broadcast like that that's something that someone want if I wanted to like I would would potentially have to write like a an extra query for it in post grass it may not be optimized I'd have to write some more indexes and that's not to me to me that's not running a calm like company like or a Dev cycle for me so I I kind of elastic is very good just throwing whatever you want at it it indexes it instantly I just have to reindex all my users and then now that's there to be uh you know queried whenever um and so I've just liked that pattern personally and it also takes so much load off the database so like our database runs very very calmly it's never querying too many users uh and it's just a calm way to run it for me um the the downside is it's expensive to run a lot of elastic noes um which you solve by just charging properly um and being being able to pay for them that's like how I kind of see it so with more money allows me going to buy more hardware and um hopefully push some more responsibility onto that hardware and I I love that Postgres elastic kind of pairing um it just kind of it works very well for for how I I run Bento at least so all these all these elastic nodes did you say three you have three of them four four okay so these four elastic nodes are they um like are they serving different parts of the app are they all um so they're serving different functional roles in the application and you're pushing different sets of data to each one yeah so let's say there's like an elastic users um uh cluster and then that has all the indexes for all my users and all of them have different mappings and all that kind of stuff um and so that one's lightning and fast I've got like an email one which I'm currently like restructuring and rebuilding now um I've got like an events one etc etc so like that's kind of like how I I split a out I I I have you each cluster has its own responsibility and I I host that on um elastic elastics managed um cloud and and they're they're great they're great it's a little bit expensive but like I think I I mentioned uh on on Twitter last week it's like a node went down an elastic engineer is paged he's on it all fixed I I go off to get my kid you know very very very nice kind of like setup there and their support tickets are so fast and and that's that's quite nice and have you done the Enterprise thing over there too so that somebody somebody's keeping an eye on all your stuff all the time not not not not so much it seems like you know there's like some Platinum support package that I have and it seems like I I get that with that so more or less yeah um but yeah back to like the the postc stuff it's just very lots of selects lots lots of single selects very lean insert and then I'm just directly selecting most of the time okay yeah I've I've wondered um cuz you and I talked a long time ago about uh storing Json in you know relational database and like how to how to query it in any which way that you want and the answer is just like it's hard man like especially when you get to you know the scale that you're at it's just it's just hard like it's doable um but at some point it starts to fall over and so that's where elastic really shines for you and so how do you get it out of how do you get it out of Postgres and into elastic like what's the actual tool or mechanism that you're using there yeah so we use a gem called um search kick which I've kind of like hard forked it's like a a rails gem um Andrew Kane built it he's like a a wizard I love like all of these gems and um yeah that handles most of the indexing I forked it and modified it so it handles multi-tenant so I can put tenants in their own bucketed cluster like bucketed indexes essentially so 100 customers here 100 customers there Etc um and gives me a lot more kind of control over that um which was like a very big Loan in for me on like the elastic side um but again the whole the whole pipeline is like user gets updated or an event comes in push that to sidekick which is a background job queue sidekick then indexes it and it's good to go it's good to query from there um yeah and then on the reverse it's like I I build a segment I grab the IDS I grab all those IDs and then I start emailing them and send them off to pyic to send all those emails on the reverse side okay so when you come when you're coming out of elastic what you're pulling out of elastic is primary keys and then you're going back to Postgres and Postgres is like yeah dude I can handle that and then you just pull all those I'm great at this yeah so Postgres goes I know what to do baby give me those IDs he grabs those IDs do and then just goes sends those emails out right um but but but that that's reduced so much anxiety for me right so so and and I I love Postgres for that it's looking up IDs and I'm sure I could obviously use like anything I could probably use SQL light if I wanted to but if you're just looking up IDs my friend I think you probably could yeah yeah yeah yeah in like other areas of the app I'm sure there's like slightly more complex um joins and stuff but for the bulk of our quaran no it's very very light it's you know lots of lots of uh yeah lots of lookups but very very very simple lookups and I just love that pattern of push St to elastic grab it from elastic
they thater throw and says hey I need this new query and I go that's actually not a bad idea I quickly add it to like the indexing so I can just say hey in the index add this to the mapping and then I just reindex all that users data and then now they can query it freely and I don't have to worry about like a a big migration or any of that kind of stuff so that that works perfectly for me those those big migrations are not uh congruent with a calm business they're just they're just not um when you're going to migrate over you know billions and billions of rows in a relational database that's kind of terrifying no matter how much tooling you have set up around it yeah yeah the the the the downside like w with this approach which has been kind of nice because post grass is now like like it's main responsibility just the hold the data and store the data and do lookups on the data I'm not like giving it too many fancy responsibilities it means now like I've begun the process of going okay well we obviously going to continue you know getting more data and eventually I am going to cap out at my Heroku plan limit right and so before that happens I need to work out all right well like what's the plan what's the strategy here and then that's now where I'm kind of in this process of investigating all right and I've been investigating for a while but now I'm starting to get a little bit more serious cuz I have a heru Heroku renewal coming up and so um I'm exploring a lot more options a little bit more intimately about all right how do I split this database like I have with elastic right with elastic I've got my users my events my emails I want my Postgres to be the same I want everything email responsibility here event responsibility here etc etc just to remove some more anxiety and more stress from My Life um and so that's currently my like big project at the moment so tell us uh so we get a sense of scale you said you're gonna top out of Heroku what is the top tier of Heroku Postgres and how much time how much head room do you think you have until it's like game over man uh sometimes next year uh probably probably like mid yeah probably like yeah no it's fine it's fine it's fine no that's plent that's plenty of time but it's it's not it's not terrible Heroku caps Heroku caps out but competitors don't cap out at at that so that that's a as an aside um and it looks like Heroku actually added a couple more plans I think um the standard plan $112,000 a month [Music] um what are they doing over there um and it looks like the pricing T is that called the Jessie plan that's eight8 terabytes so so that's that's yeah yeah yeah so you know um yeah so that that's that's the max but regardless I I want to I want to split out from that um and the the solutions that I'm I'm looking at is either I go to crunchy crunchy is great because I can potentially have these multiple databases and they allow me to scale as it should be RAM and storage and because I'm not doing such expensive querying that works so much better for me whereas Heroku they're like no you need the big boy everything right um and I don't need the big boy everything um and so crunchy gives me that flexibility the other option is I've also got citus which is the post grass extension um and I I've looked at going to my Sequel and stuff I'm not totally keen on that after like a little bit more research I I want to stay in the Postgres uh world really you looked you looked at moving different really tell me why did why did you that even cross your mind because you wanted to use something like vitess yeah so I I looked at V a little bit um and like everything kind of looks nice I think now it supports foreign key constraints I think it does yeah last time it it does now at the time of doing the main providers didn't so I I wasn't totally interested in like dropping all my fgs MH um but also I'm not like totally sure I really need to I I just don't want to learn my Sequel and like developed such like an affinity yeah I've Dev developed such an affinity for post grass and I don't like seeing seen the The Vibes are important to me this is going to be like a weird CH but the Vibes are important to me and when I look at my sequel I see things like people complaining about performance issues on like social and everywhere and like you know more upgrades come out and there's more degradation in it gets worse and it's kind of like I don't know for as Postgres it's like everything's faster everything gets Rel everything's faster and the numbers are so much bigger and I'm like I love big numbers give me those big numbers um and I also find that um a lot of the contract actors and people that I speak to in the Postgres world are a lot more approachable to me um and there's a lot of rails Postgres people uh that I've been able to talk to and they've been fantastic and helpful that and and that's my general approach if I run into a problem I try and find experts to talk to and if I find a lot of experts that are very approachable I'm obviously going to be a little bit more pulled into like that direction you know so yeah Ys y's ecosystem uh the rails ecosystem is very heavily Postgres and you make an interesting point about the like the experts being approachable I could name 10 or 15 Postgres experts on Twitter that I could DM right now but there are very few MySQL experts that I am aware of um many of them work at at a company I used to work at but there are a few that are independent but it does seem like it does seem like there are a lot of Postgres people out there uh willing and able to help pretty quickly yeah even like the crunchy data team like I've messaged Craig a bazillion times I've jumped on calls with them and stuff and they're very very they just help you and I think they're a little bit more longterm so yeah I don't know the the energy feels a little bit better in in the post space so I'd rather stay there the cus stuff is really interesting to me um have you done much research uh into C I've done a little but yeah explain and I actually had Craig on the last one we talked a little bit about cus but explain for the The Listener what cus is and maybe what it compares to yeah sure so it allows you it's a Postgres extension it allows you to basically have a whole bunch of Postgres nodes that all work together and then you can have what's called I'm pretty sure it's called a distributed table uh if I recall correctly and then that allows you to give it a Shard key so let's say for me it's like a tenant ID or a side ID uh and then your data kind of gets spread across them um and then you can Auto balance and all that kind of stuff you can spin up new notes nod and it just allows you to scale a little bit more easier and it means if a node goes down then you can bring that one back up it's not like the end of the world so it allows you to have distributed tables which Vitas and all that uh allow as well but it remains a Postgres extension and um you can get a Cloud solution with Microsoft uh with the Azure stuff oh really oh with Azure it's super affordable yeah with Azure they call it some horrific name it's uh it a hyper cloud or something like that hyper Cloud I don't know it's something hyperscale hyperscale that's it hyperscale cus is now Azure Cosmos DB for Postgres oh so it was it was hyperscale and now it's Azure Cosmos of course no that's perfect that's perfect yeah good good idea good name good name but but then the issue with having a large single node is uh you know even if depending on like your backup strategy so back we use continuous protection with Heroku which is just snapshots regularly and then you've got the wall and so bring that node back up it restores a snapshot replays the W you're back and then it runs um it's a lot easier to get multiple nodes up uh simultaneously than a single node and so a lot of the sidus benefits is just being being able to you know if if you go down raise them back up really really fast um I kind of I like I I like that structure a little bit more than having a single node but at least at this scale I may consider uh as a first step just having multiple single like single nodes have their own responsibility um and then kind of go from the so you're considering yeah if I'm hearing this correctly you're considering potentially two different um sharding strategies one would be more traditional where you're sharding on a tenant ID and so some n number of tenants go per you know Postgres and then you know you send your queries to the right tenant based on on their tenant ID the other one would be your sharding based on like not uh email domain but like domain in terms of like the application term so like emails go here events go here users go there and so talk me through this like this thought process and where you're currently at where where I'm currently at is I I like the single node approach because once I actually break down my DB the total DB sizes actually aren't too big um and some some of the issue that I have actually with Postgres is there's there's quite a lot of bloat in there um I think from just updating records right cuz whenever I send out an email all those users are getting updated anytime an event happens that user is also getting updated so um I could probably like tune vacuum a little bit but it's never going to reclaim space until I do a vacuum full and frankly I'm not going to do a vacuum full because that's going to lock the table um and so a part of this process actually just want to like move to another DB so then I could potentially vacuum those ones independently or or rebuild yeah on the way out or rebuild them or an approach that I've considered uh and I actually got this idea from uh when I was exploring like the vest stuff um is potentially using rails just to double sync so just write so uh send and then have like a background job just push that to the other separate DB and then I can run a background job just to fully sync all the backlog over a time period that I'm very comfortable with um and then that other DB is then you know it's getting those uh inserts fresh um and so it'll potentially be less bloated and all that kind of stuff so I've got a couple approaches another approach I've got is like uh potentially streaming in an exact replica from uh One DB to another DB uh and then just changing reads and writes so let's say I'm I'm going from like a let's say the emails table I've got it's all going to the main DB I can get up a replica switch that to be able to write to and then just in rails just toggle it over to start writing to the new one right so that's also an approach yeah there's there's a few ways I can do it they're not yeah I'm just trying to pick which one I Vibe with the most and it's least stressful and which one do you Vibe with the most like right now which one are you feeling I Vibe with um I actually Vibe with having multiple single nodes on crunchy um because I I like the idea of on the rail side just being able to switch over my reads and writes and coordinate with the crunchy team just to get that um one straight on and I can enter a maintenance period on our app queues up all those background jobs switch over the rights reboot all the instances and then I'm kind of good to go and then on that new instance drop all the other tables CU I I I don't I don't need them anymore right and then back up and run in you know what I mean so I I can have a full replica of both the instances I'm running kind of like two core switch it only for a couple of tables drop everything else and then I'm I've got like a nice clean table or a nice clean database with just what I need uh that's like the least stressful route I think um
yeah why are you looking at me like that I didn't I didn't know if you were going to keep fiving you you looked you stared off into space you said I think that's the least stressful route and then you said yeah so I didn't know if if you were second guessing your your decision no because because a lot a lot of it's you got to kind you just got to explore this stuff right um and you got you got to do it a few times and and uh you know mess up a few times because this has been my experience trying to get elastic to work actually like just any database you just got to try a whole bunch of stuff um see which ends up being the most seamless and then when you find something that is pretty seamless then um yeah yeah then roll it out and commit to it but I I try not to like give myself any like really hardcore deadlines and just take things nice and slow um but I do like having multiple sing Single nodes yeah that that's what I want to go with I'm G to put that in yeah there done well so I we got we got to talk about this because you said earlier that you know when you were working with Andrew you're like yeah I was you know I was learning to code kind of on the side so that was at most six years ago and and now you're now you're a one a one man show running an ESP with your own mail servers and you are migrating massive databases what how are you like talk to me about like your your approach to learning things or like how you've gotten to where you are now because this is a lot of very complicated stuff and bu your own admission six years ago you were like ah I'm just trying to learn the code so how did we get here I don't know man um I don't know man there's probably so many like easier businesses to run but in saying that like in saying that like just let me let me think about about this it is it is it is complex but it's also it's also not right like at the end of the day Bento is kind of like a glorified app we take in form submissions we process them we put them in a database we have some call backs that send emails that's actually not too too tricky query is not too tricky um elastic is like a beautiful has a beautiful DSL is that what you call it beautiful querying language to um to to hit to bring in data um any problem that I faced touchwood I've always been able to overcome even if it's been horrific and catastrophic I've always I've always figured it out eventually um and I think having done enough of those where I can figure it out I have the confidence that I will figure it out um but I also have a little bit of a change in how I used to approach things so I'm a lot slower in approaching things I have the budget now to just spin stuff up um and so I'm comfortable just spinning stuff up and trying stuff um and so when I'm working with like a lot of data yeah there's sometimes no other option than just like rebuilding the world playing with the world a little bit and uh if it doesn't work just blow it up and try something new right and I think that that that approach works works pretty pretty pretty pretty well with me um but when I'm making all these like decisions and where to go I am very heavily biased of like where I can get help um and so like on the in terms of my decision for all right do I go to cus or do I go to crunchy with multiple single nodes I'm a lot more biased to go to crunchy because I know that if I run into a scary hairy problem I have someone to talk to and I can jump on a call with them and I can just like riff and that alleviates so much like I remember when I hit like one terabyte on Heroku I was like panicked I was like oh got too much data um just cuz i' never seen a terabyte database before yeah yeah it's like scary um and I'm like I don't know what to look out for like what's transaction wraparound what's like all this kind of stuff right uh and so I jumped on uh a guy called Jesse at Heroku who's now a crunchy apparently um they've taken everyone they've gutted that business out so I I I chatted to Jesse and we went through my instance he told me everything we talked through stuff I learned uh and like through that process I'm now confident in a whole bunch of things and I'm also not paranoid in a lot of things I think like with databases knowing what not to be scared of is like one of the largest challenges being like hey I actually don't need to worry about that or if I change things I don't need to worry about that either um and so a lot of my design decisions are going how do I just not ever worry about this silly thing and then I'll try and restructure things so I'm never worrying about that silly thing same with like who my database is exploding all right well how about I don't store HTML content in the database why don't I put that in S3 like maybe that's a smart thing to do right um especially when someone's sting on 100,000 emails don't do that so just just decisions like that take a lot of the the weight off um and I think that's been helpful being able to run a business at least of this size and this scale uh calmly if that makes sense um yeah it makes a ton of sense so I'm I'm hearing a few things um as it relates to being able to run this what what in my opinion is a very difficult business to run even if it is just crud it still seems very difficult uh one is you've you've made a lot of decisions optimizing or at least with an eye towards having a calm business life and I think that needs to be stated because a lot of people are like oh I would love a calm business and then never make decisions that would make their life calmer or easier so that seems like number one uh at least like consider it when you're making a decision number two is you now have um I think from what you've said you now have budget which is margin to like try some stuff out so you can you know you can get a whole replica of the world and mess it up and then burn it down and that's totally okay and then the thing that you keep talking about which I think is very important is um your relationships in the industry like people that you know like and trust that you can reach out to and either just have a call with or explicitly High to to help you do stuff is there anything else you think like do those feel like the three main things towards like that have allowed you to have a calm business at this scale yeah I think so what one can I give like an example of like a product decision that that's help so um for example in Bento there's obviously a lot of data and a lot of pages load a lot of that data um and sometimes even on like a page like the let's say the visitor browser you know when you go to people and you see all those users sometimes that page can have a lot of n plus ones you know we got like tags loading Fields loading sales loading just a lot right um and for Postgres that's not too hard to do but sometimes that can be slow um and so what I've tried to do is I've tried to remove a lot of the querying out of the web request so when you load when you load that page uh I actually created a websocket connection and background job in in a background worker that builds the world and pumps HTML back into the page I don't actually ever that's why Bento is so fast so when you're when you're going from page you probably notice Bento is very very fast it is but I'm actually never doing a request other than like a you know site ID user ID all that like if I if I if I showed you my querying every web request is like instant because almost none of it's happening in the web request cycle all of it is happening in background jobs most of it is happening in background jobs um and that is allowed n plus1 being in a background job in response to a request yeah yeah yeah I mean you can go to it now if you go to Bento it'll have like the spinner that has opened a websocket connection background job gets processed and then we're pumping HTML back into the view um and that is why Everything feels so SN happy um so when I when I come back to Bento after leaving a tab open for a long time it has this notification that's like websocket thing has been lost and I'm you know leaving it open for 24 hours what's he using websockets for everything is the answer everything you you so so and and and and like another thing is like you know when you go to flows I do think I I got a Postgres query for this you know when you go to flows and every node has stats on it right yes and so I I obviously have to do I obviously have to do a count on that now if I had to do a count on the web request when you loaded that page it would exp came over what happens if someone has 2,000 nodes on there which some people do right users users will be doing stuff so you know that somebody has 2,000 nodes and that's the that's the thing so users will always be doing stuff and they'll always be breaking things in like the weirdest ways and so with that one what I did like let's say now talking about flows with flows I have it so we uh I'm using like a react Observer um uh library and when the when the node comes into view I create a websocket connection that then joins a background queue that does the SQL count and then it streams it in um and so a lot of the the product design decisions have actually been around around me being mindful that okay I want to be like a little bit more generous on my Postgres instance and I want to allow it to have time even if a query is not optimized and so I take that out of the web request put that in a background job and then that allows things to be perceived be lightning and fast even if they take a couple of seconds to compute because if you click a button and it takes 2 seconds that sucks and that happens in every other email marketing provider that I've ever personally used because they're doing it in the web request and so it just gives you a lot more who could blame them everyone does that Jesse everyone but you has nobody's figured this out so what do you send me when I click a button what do you send me just a shell just like an HTML shell that you then pop late depends on the page so like um it dep it depends on the page so like uh let's say the tags page people only have 100 tags querying that's very easy so there's no websocket for that but there there's a a counter on the right hand side of the tags each individual tag has a count of how many people have the tags that's tag. user tags. count or whatever that is yeah that's a websocket request I'm always going to websocket request that one cuz I don't know maybe the counts slow or let's say um someone's doing a lot of Rights there's so many things that could go wrong and so I always I always push that to yeah I always push that to a web socket and then stream in the HTML so have you built like a little kind of like a little homegrown framework to say like on the front end what is being requested and then on the back end like H how do you tie all this stuff together in a way that's not bespoke for each each time yeah yeah so it's using action cable so it's and and it's very lightweight code so I like kind of create a connection to action cable and then I use like side kick effectively to create like a background job and then I stream a partial or some Json or whatever um and then that actually populates I don't even know why this like I feel like this should catch on cuz it's very effici should catch on there's rules I love this yeah it's very very handy like and it it again it takes like for example let's say an elastic cluster goes down like my whole app doesn't break and I can actually also stream in an error response to say hey we're struggling at the moment and have like a spinner there so there's there's other benefits but especially for those like n plus 1es n plus ones aren't inherently bad and you you sometimes can't optimize them away a lot of the time or sometimes there's like not even any point to optimize them away so you can just load the content a little bit different or think about loading it a little bit different and um I found that to be another calm way to kind of uh on the product side to to make Postgres a little bit easier to work with by just yeah streaming stuff through background workers just giving it a little bit of time I like that this is this is exceedingly clever I want to steal this and bring it to laravel in some cleverly named package yeah stream stream stream blade stream blade or something like that stream blade that's a good you know we'll Workshop it a little bit but that is a good first that's a good first step I like that no that's super clever I didn't realize um I didn't realize that that's how we were doing it I think the the combination of a websocket and offloading it to a background job is wild to me cuz most um cuz like what what I very early on before I started using elastic this was the issue this is where I actually started doing it when I when I was predominantly on post grass stuff would would be occasionally slow there' be query in a custom field and it would be slow and I hated that the page would take 30 seconds to load and and Heroku has a 30 second timeout Heroku has a 30 second timeout right and humans have like a 4 second timeout nobody's waiting around humans have much less than four so um and so by re architecting I gave myself some Grace and a little bit of breathing room and then I could obviously work out stuff um after that and I obviously restructured and refactored stuff um but yeah trying to take stuff and responsibility out of the web request uh has been very very very very very very helpful um yeah especially when bringing in a lot of data or like let's say uh like a visitor profile page doing user. events right like even penting through some of those someone may have like 10K events and like no matter how much I kind of optimize that pagination it's too it can be very very tricky um and yeah not not optimizing it it's like sometimes okay if that takes there's a spinner for 3 seconds you know like that that that's okay and it's not too much stress on the Postgres instance but having the page take three seconds n so no that's no good yeah I think my advice on the product side would just be trying to like re architect your product in a way that like you know give yourself some Grace you know maybe push stuff off to background jobs or push stuff off or just change how you render stuff um so it's it's not so much of an issue Okay so final question on this because my mind is blown the when you I don't know much about rails but so you have this active cable action cable active cable action cable action cable so you have this action cable concept do you hand off some sort of unique uh uh like action cable instance ID to the job or how does the job know who to phone home to mhm so um so action cable creates a connection I think they call it like rooms so kind of creates a connection there's now a persistent connection from like uh the client to the action cable server so that's that's unique and action cable keeps a lot of that like it handles all that kind of stuff okay then from from there I can then uh I kind of create like a unique job ID on the client side um so it's just like a random hex or whatever uh and and then that creates the job ID the job then gets processed and the the job has knowledge of like the job ID and the action cable user and then I stream it back in and then the user gets it to to populate this also works really well with like AI type stuff because some of those take a little while so if I'm doing like a compliance check on an email it's okay if that takes 20 seconds to load because it's um happening in a background job not a web request um and so that that's super useful as well yeah so yeah action connection gets created job gets on cued then job streams down a partial or some Json to then render uh in the app so yeah again with this theme of making my Postgres calm by taking all all this like it's all related it's it's it's all related um just taking a lot of the anxiety off me and being able to build super fast apps where the web requests have very low responsibility on the post CR side it's funny because it seems like everything has low responsibility like the postest has low responsibility the web requests the background jobs just doing a little count elastics just you know you;ve got four different elastics they're all just doing their own little thing it's like who's got the responsibility here it's all little responsibility all the way around not you it's great I love it this is fascinating yeah it's it's different right um and yeah yeah it's it's different so so Redis it's like action cable has its own red this someone's probably going to call me out on how dumb this is but action cable has its own instance rails case has its own instance sidekick has its own instance our rate limiter has its own instance now is that Overkill yes is red as cheap not on Heroku same price it was 10 years ago same price it was 10 years ago but um but relatively it's cheap right you know talking uh you know a couple of customers per rdus instance uh means that I at least have that reliability and I'm moving in the same direction for Postgres so this has being my theme my theme is is that I end up trying to I start with one I started with one elastic instance I started with one r as instance and I start kind of breaking them down into much smaller responsibilities um and then that makes the problem set a lot easier to solve I'll probably go one of the reasons I'll go to single Noe like multiple single node instances post is that'll be step one and then maybe the emails one becomes gigantic and then I just have that emails database to deal with that I can then go hey you know what I'm G to move that to citus or hey I'm GNA move that to to Vitas because that's a good next step for me um and it just it it just makes things a lot easier to manage right whereas if I just have one big monolith of any of these it's like oh oh boy this is a little bit stressful if like let's say on the red side if I get dos all right well now my Sid Kick's down that's not great um not great no that's not great or now people can't load uh I I had I had this one user who left a book on their re refresh button on their computer um that is some Silicon Valley stuff right there massive account massive account right complex elastic query on the on the people us this page just like refreshing right hammering and um just Ju Ju Just hammering right and they're behind the offall they're inside the castle they they're inside the castle right and so I I ended up using uh Cloud flare to make sure that never happens again um but uh dur during that happened that that that uh that backlogged uh what did that do that that killed my action cable redis instant which then which then meant things in the app didn't load not great but because sidekick was still up and everything else was still up um basically no one could like there was Spinners but our app is like people do stuff and then they leave right and we're sending emails automatically on their behalf everything else continued to run smoothly yeah so so having things split means the problem set by the way when I was doing that I was like at a best spy on the trip to Canada from Japan with a 2-year-old that had just woken up um but having all these small things it's very easy to like you know just restart a single instance and all that so a little bit of a a tangent but at least at least fantastic how great story we love a good story so the only I think the only monolith I'm hearing is your rails monolith and you mentioned early on you've got a fast uh fast Ruby team working on that so let's wrap there what's what's going on on the rails monolith side yeah so um on the rails monolith side so rails 7.2 I think so rails 7.2 and obviously like rails 8 and stuff um comes jam-packed with really great uh stuff for multi-tenant DB um and so I want to get my app up to like the latest on at least on seven just so I can pull in a lot of that stuff just to make my life a lot easier things like joins across multiple databases all that kind of like good stuff um and by being on the latest version it just allows a more seamless transition over um and then once once that's done then we can start you know experimenting by rebuilding the world and uh seeing how that kind of you know seeing how we're going to orchestrate a lot of that um but yeah so rails rails 7 comes with a lot of and from my understanding a lot of it's pulled from GitHub like how GitHub does multi-tenant DB so I think GitHub has multiple databases for all kinds of things and a lot of rail 7 is kind of in their philosophy of doing things from from my from like what I've what I've read um and yeah I'm probably going to follow that path for for for the the most side of things um so yeah wow so cool this has been a delight is there is there um so two things one anything else on the technical side that you we haven't covered that you want to cover no no I'm I'm no I think I think I think that that that's most of it my my like my my big kind of uh I guess ask for anyone that is trying to like build something is like don't over complicated you can stay on a single node for like quite a long time it's very rare that people are storing like as much data as as we are and from my understanding like talking to some of the um yeah like some of the post experts people are running very large multi multi multi multi terabyte uh single node instances kind of pretty calmly and well and a lot of that is like trying to it's just like how you use the database so your database can get quite large if you use it in a certain way and so just trying to work out all right like how how do I want to use this database so there ways that I can use it that are less stressful and are a little bit calmer and then try and like design your app around that and a lot of the times when you do take some responsibility off the the database I don't know you sometimes you build better products because of it I think Bento is very very fast I think it is like quite delightful to use and a lot of that is because of the decisions I've had to made because of post grass um try try to make it like a a better experience for me and that's led to a better experience for my users I don't know if that makes sense but it make feels right to me yeah no that that makes perfect sense um okay so the last thing I want you to say is give us your pitch for bento but before we do that um you are not out on Twitter doing a lot of like thought leery stuff but you are a one person bootstrapped company profitable calm so give us your G give me give me your pitch to The aspiring the the young Jesse Hanley the person that wants to be in your position running a software company without losing their mind without having to raise VC if they don't want to how how would you encourage the Next Generation of of aspiring uh Builders that's a big responsibility um okay so like I think I think something that has been true to me and I actually mentioned this to like another guy on on a demo call who asked me like my advice is like you got to stay true to you know like when you start out you've kind of got those like rose tinted goggles of like how a startup should be and how a company should be you know you follow base camp and all that mhm you have to stay true to that and sometimes that sucks and sometimes it's actually not too much fun it's not fun B trap is not really that fun you got to like take money from your own pools of cash um sometimes you got to pay yourself not that much for a while um it takes a really long time um like I'm seeing I'm seeing one of our competitors that has a a bug as their logo you know they're they're multiple times larger than us um but I I don't think they're profitable and it sounds really stressful in there right and I'm like well I could potentially be like that I could raise some money um but I I don't want to like I love living in Japan I have a 2-year-old daughter who I absolutely adore I've been able to be home for all of this time um I love my wife to bits I'm able to go get lunch with her whenever I want I can chill like it's it's just it's a good lifestyle but I've stayed committed to what I wanted and I've been okay with the tradeoffs of what that's entailed right like I've been I've been okay with not growing that fast I've been okay with taking things slow um and it's turned out it's turned out fine for me so I think like if you have Rose tinted goggles of like the business that you want to build try not to make compromises be like super patient with it um listen work obviously very very hard I've worked very hard on this business very very hard every day I work on it and uh and it's paid off it's paid off for me and so I think hard work staying committed to your ideals most often figures itself out if you kind of stay in the game long enough enough that maybe that's the the one caveat I want to put if you stay in the game long enough you'll hopefully figure it out otherwise maybe try something else G to go for six years and and haven't found product Market fit maybe you should try something else but yeah maybe pivot a little bit yeah yeah I like that that's a great answer I I put you on the spot there that's a great answer thanks I know that uh I know that you want to lead a quiet life and work with your hands but we could use more people like you being vocal on Twitter but I won't I won't put that on you that's that's a lot you don't want to wait into those Waters yeah I mean like slight slight tangent part of not I don't want to be a thought leader and like I don't actually want to attract too many customers like this this is a weird this is a weird thing I don't want to attract too many customers like I I love that you're a customer I love that some of your friends are customers I love that like a lot more of our friends are trying Bento out and and using that and the company is profitable and like frankly like I think I mentioned on on the Preamble before we kind of went into this my rent's like under 1,500 a month including utilities and stuff I live in Japan food here is like I can have an amazing like tesu meal for like5 to7 I don't I don't really need too I don't really need too much more money our daycare is $4 an hour we're talking about daycare before this um you know how much more money do I need so like a bunch of people just cursed at you just screamed out loud at you for that one yeah so it's like after I paid myself like what I was earning uh when I was running the agency I I don't I don't really need too much more than that so um yeah but it it did take it took it took I've got it in the footer of our website it took to get to this point 6 years and 251 days oh wow we're almost at seven almost yeah almost said seven it just it just takes a really long time well you are uh you're doing incredibly good job and if you're not you should be proud of yourself um because this is this is very inspiring for me so tell us uh I know you don't want too many more customers but tell us uh why people why people should at least consider Bento yeah so um Bento is very good at a couple of things we're very good at sending marketing email we send all your newsletters and all that kind of stuff we're very good at sending an automated email so someone signs up to Aaron's uh weight list you say hello I'm Aaron you're on my weight list we're very good at that we're also very good at a couple of other things in that automation area um and then now we're getting very we're getting better at sending transactional email so in the last two weeks we've actually you know how you hacked on that mailer for laravel we now have an official laravel SDK with that baked with that baked in you can now say like whatever it is you can do the MAA Bento thing and the transport that's what that's what it is right yeah the transport and it all just works and then we've got all these nice Laravel methods we shipped a python Library we shipped uh a new rails Library shipped a swift library for funsies we shipped we're going to do a Unity Library so we can serve video game developers I don't know it's just fun um there's there's a guy there's a guy in uh called Zach I actually want to give him a little bit of a shout out there's a guy in Zach called Zach uh who lives in diu which is some neighborhoods away he signed up to Bento as a customer made us the Lille SDK and has just kept hacking he doesn't stuff he's he's a mad man he's like I I built a swift library for funsies now we've got a swift Library so um Zach's being Zach's being a machine I uh yeah I just want to give him a public shout out so good job Zach yeah he's he's he's been a he's I am paying him for his work but he's just he was like a customer that wanted to like just help out for fun um and so yeah he's been paying him to build these libraries and so we got sdks coming out the wazo so uh that's been that's been good so for most for most things people you know nextjs and all that stuff um most Frameworks people are building on that are watching this they're dealing with databases uh bento's got very easy ways now and much better docs to get started to send marketing and transactional email so that's the pitch yep and I I coign that I use you for marketing and transactional so we do all of our password reset through Bento obviously we send all of our uh SQLite Postgres aaronfrancis.com screencasting.com through Bento and just I think yesterday or the day before uh I was dming you about pulling down tags from the API I use your API for everything I'm pulling down I'm syncing down all the tags so I know locally when somebody like somebody from SQLite signs up for Mastering Postgres I can look and see like what tags do you have all right let's let's do a sync job to make sure you have all the right tags in Bento and I can keep them locally it's great I love it I love it it's great great product thanks for saying that and thanks for being a customer it like puts for my family so I appreciate it thank you of course yeah well um thank you for doing this uh I know that uh I know we've been trying to do this but Japan and Texas are not lined up uh on time zones quite as easily so I'm glad we finally got to make this work this has been a delight thank you uh thanks for giving us all of your knowledge and where can where can people find you on online uh on Twitter we've got Bento Bento the handle um and so that's that and then I think I've tagged my ego's tagged myself in the bio so you can type in my name it's Jesse T Hanley you can kind click through there but it's a lot easier to remember Bento on Twitter um and then yeah and then Bentonow.com to start a trial I don't know okay giving them the hard clothes Jesse you can talk to me this way um maybe like a maybe like a give if anyone wants to like just chat about stuff um or chat about like database stuff there's a there is a demo link just tell me that you you don't want to be sold and and I'm happy to tat about database stuff I'm happy to talk about scaling rails I'm happy to talk about anything that like we've covered in here even just as like a friend cuz building apps is hard building apps is stressful and it's a lot easier when you know there's approachable people out there to talk to and so open invite for anyone to click the demo link and just tell me ahead hey this isn't about Bento I just want to talk about rails and I'll happily talk to anyone for 30 minutes about whatever problem they've got so yeah there a awesome if anyone what's up so yeah so generous thank you all right y'all well um if you like this episode please either I guess subscribe on YouTube uh aarondfrancis or there is an RSS feed somewhere uh it's in the show notes so go down there and find that and I'll put links to everything Jesse uh does and has mentioned in the show notes and until next time we'll see you later bye