Various Issues on Configuration a BizTalk Installation.

Bit of a hiatus since my last post, but life has been pretty busy lately. Started a new job recently (yay!), so Im currently settling in which hasn’t left me with much time for personal development.

I’ve been asked to do a basic presentation on BizTalk, introducing my new team to its basic concepts and what its used for. Im currently in the process of setting up a couple of demos, and (as always) BizTalk is giving me fun and games getting the darn thing set up. Again, as always its my fault rather than BizTalk’s, so here’s a couple of things I’ve just run into which I thought might help someone out there.

The first of these is the following error:

Failed to generate and backup the master secret to file: C:\Program Files\Common Files\Enterprise Single Sign-On\SSOC7B0.bak (SSO) : Access Denied.

The cause of this is (thankfully) quite simple: the account you’ve just configured BizTalk to use doesn’t have the necessary privileges on the machine to complete the configuration of the SSO component of the BizTalk installtion (provided you’ve selected SSO as one of the installed features). So once I’d added my BizTalk user to the Administrators group, that error message went away. Interestingly, when I went to re-configure the BizTalk installation it warned (!) me that the account had administrator privileges and posed a security risk to the box. So this makes me think the BizTalk service account doesnt actually need admin rights, but instead a particular subset of permissions slightly more elevated than normal. I dont particularly care right now, as I’m just trying to get the bloody thing up and running, so thats a post for another day 🙂

The second error was surprising, and was one of the validation errors BizTalk scans for when beginning a new configuration:

SSO DB already exists [this isnt the exact wording as Im trying to do this from memory].

Ok, so this part is really important: Restart your SQL Server Instance. Until you do this SQL doesnt play nice with the SSO DB (or more likely the newly created SSO Windows Service isnt playing nice), so restarting the SQL instance clears any crap left over from your attempted install. That should then let you drop the DB to avoid any further issues, otherwise there will be services that are holding connections to the SSO DB from the failed install.

The third (and last) of these errors was:

Failed to generate and backup the master secret to file: C:\Program Files\Common Files\Enterprise Single Sign-On\SSOC7B0.bak (SSO) : Access Denied

Look familiar? Yep, exact same error as the first one, but the cause of THIS was different, and far, far more annoying. I actually got to the point of uninstalling BizTalk several times (along with Enterprise SSO), multiple reboots, SQL instance restarts. Nothing. It kept failing on the same point (ie. the start). So I started looking a bit deeper into the log file, and found a permissions error related to the local account I was running on (not the BizTalk service account), saying access denied. “Strange”, I thought. “Im an admin on this box, BizTalk even allowed me to restart the WMI Service!” (this is amusing to me, because trying to install BizTalk on my local work pc was a bloody nightmare… thats a story for another day).

Then I had an idea… when I originally began to configure my BizTalk installation my service account it spat an error at me saying I couldnt use an account to configure BizTalk that had no password. So I created a password for it, and it went away. I realised at this point that Server 2008 r2 was logging onto the main account WITHOUT a password. So obviously, it must realise that my local account didnt have a password, and as such wasnt playing nice with SSO (hence why BizTalk spits an error when you try and configure a service account for it with no password). Would have been nice for BizTalk to tell me that at the start, but its ok. Once I set a password for my local account (which, silly me, I should have done from the start), the configuration worked perfectly! Ive now blogged this hellish issue, so it wont be an issue anymore 🙂 (or at least as big of an issue haha).

Hope this helps some poor BizTalk dev out there.

Advertisements

Asynchronous Programming in .NET 4.5 (Video)

Michael Kennedy on Technology

Here’s a recording of a webcast I did for DevelopMentor covering .NET 4.5 and .NET 4 asynchrouns programming concepts, including Task<T>, async, and await.

Enjoy!
@mkennedy

PS – Be sure to watch it in 720p quality.

View original post

Challenge Accepted

Thought I would share an interesting problem I recently had at work.

Our client has a collection of InfoPath forms it uses to connect to web services for testing BizTalk interfaces from external systems. On the whole, these are pretty stock standard affairs, the relevant BizTalk orchestrations are exposed as Web Services (through the magic of BTSWebSvcPub) and the InfoPath forms (prettied up for client usage of course) submit test messages. This enables them to do pseudo end-to-end testing without having to get half the world involved. Amusingly (amusing because it only just occurred to us), a few days ago I realised we would need to upgrade these forms in line with our BizTalk 2010 upgrade project.

Let the games begin.

As far as things go, it was pretty straight forward. I refreshed the web service references and checked the forms still compiled. So far so good (well, minus a few bits and pieces cropping up, but not relevant to this post). And then came the testing of the forms, and by that I really mean… *head-desk*.

When I clicked the “Submit” button… nothing. Absolutely nothing. Oh Infopath was happily assuring me everything was ok and working awesomely, but on checking BizTalk (and the DB of the relevant application), absolutely nothing. No messages were suspending (BizTalk side), nor were there any validation errors (application side). So a bit of head scratching, a bit of googling and this left me with a lot of issues completely not relevant to my problem. On a side note, googling “vanishing web service requests” unsurprisingly doesn’t yield too much useful information. I also scoured the event viewers of the BizTalk servers and application servers (I even got desperate and checked the event viewer of the machine I was calling the Infopath forms from), all to no avail.

Then I checked the IIS logs of the machine where the web services were hosted, and I spotted a HTTP code Id never really seen; the “202” code. This is the “Request Accepted” HTTP code and means that the server has accepted the request (Yay), and the request (this is important) might or might not be acted upon, as it might be disallowed when processing actually takes place. Initally I thought “Awesome, so its definitely hitting IIS and getting through the web service, so WHY OH WHY isnt it getting through to BizTalk?”. So some more headscratching, googling and *head-desking* later, the penny finally dropped.

Accepted != Ok.

The HTTP codes 202 and 200 are completely different (duh), meaning that while it DID hit the web service successfully, the fact nothing happened in BizTalk afterwards means something must be stopping it in between when it hits IIS and when its supposed to hit BizTalk/MessageBox (keep in mind BizTalk is just a bunch of SQL DBs that require the correct credentials to connect to). This was why I wasnt getting any suspensions in BizTalk, it denying the credentials supplied by the web service as they didnt exist in the DB! So on checking the service accounts that these orchestration web services were using, it stuck out like a sore thumb… I had configured the web services to connect to outside entities using our normal dev service account. While this is ok for our local dev environments (as its what we mostly configure our web services with), I had forgotten that these web services were a completely different beast, and as such should have been configured with the default service account the rest of the IIS Application Pools were using!!

So once Id reconfigured the application pools and fired off an Infopath message, they hit BizTalk (and then the application) successfully! Huzzah!

While this is a bit of a *facepalm* moment for me, I thought it was interesting that in terms of actual ‘error messages’ there were none immediately visible, only the symptom of a vanishing message. It made me look a bit deeper into things than I usually would (in this world of “GOOGLE ALL THE ERROR MESSAGES!!1!1”), and was quite satisfying when I did eventually figure out what was going on. Thinking about it now actually, I think if I checked the DB Server event viewers Id probably find a few Audit Failure events. Hope this helps someone out there with a symptom of (seemingly) vanishing messages 🙂

Night all!

Writing Is Hard

One thing Ive come to notice since I started this blog, is that writing is bloody hard…

See, now I really enjoy writing. I find myself able to express myself quite clearly through the written word and since starting this blog I feel my writing (both technical and non-technical writing) has improved and I (like to) think some people even enjoy reading it. Writing is fun, because alot like other art forms (of which I consider software development one) you start with a blank canvas/page/screen and you create something.

But, again, writing is hard.

Ive had quite a few posts where inspiration hits and Im madly typing out about 1000 words of writing in 45 mins and at the end I feel pretty darn good about it. I come back to the post a week or so later, look at the post and just think ‘um… this is kind of crap’. I find grammatical errors, the odd spelling error and (to me, this is the worst) Ive expressed myself poorly.

It seems that, alot like code actually, the first cut is always pretty crap.

I suppose thats why writers/authors/poets go through several drafts of their work before they consider it submittable. Ive recently being doing a bit of reading on a site called Joel on Software. I find myself inspired by some of his posts… the quality of his writing is a cut above what Ive read so far on the internet and I honestly believe its something we should (bloggers or not) aspire to. This is how I want to be writing.

Im going to be dedicating a bit more time to my blog from now on… The aim is to set a certain amount of time aside per week and actually plan out my posts. Throw ideas around as to what Im going to write, and most importantly do a few drafts before I hit that shiny magical ‘Publish’ button.

If you feel so inclined, share your writing experiences below and any sites you consider the author to be a darn fine writer.

May the (Open) Source Be With You

Ive been thinking alot lately about work, and how much it can take over your life unexpectedly. One moment youre enjoying your days (weekends, even!) and the next you are neck deep in support tickets, code changes and project work. While this certainly doesnt make for a boring work life, once things really get intense you start to forget how much you actually enjoy programming! It becomes a chore, a case of ‘sigh, another day *grumble grumble*’. Ive realised this over the past few months that I have been completely consumed with work, and in doing so stopped enjoying the very reason I got into programming in the first place…

That urge to discover, create and mold what you see.

Damn, its been that long since Ive actually programmed for the pure joy of it its given me a yearning (oh yes, I just used the word yearning) to work on some personal projects of my own… Nothing too big, more ‘for the fun of it’ type stuff than any serious applications. And as a result of this I have chosen to join the open source community, in this particular case GitHub. GitHub is a site which encourages the collaboration, creativity and passion for programming (all shapes and forms) and enables anyone to contribute to any project that catches their eye. These are people who, apart from their daily grind, choose to spend their spare time creating something wonderful, with people who share their passion for development. This sounds like a mighty fine place to be.

The smallest ideas often cause the greatest change, and I find the best example of this with the guys over at Code52, and they have a very simple goal; ‘A new coding project every week’. While they havent quite met this goal, some of the applications theyve created are amazing, with the base code created within an entire week. I find this sort of thing staggering in the most wonderful way… Most people I know (even people in IT) consider their job as ‘just a job’. While theres nothing wrong with this, for me personally I have a passion for this sort of thing. Not just the technical side of it either but the reasons why we develop (to me, this is almost more important than what we do but thats a post for another day). They are trying to introduce the open source mindset in small, easy to manage projects. This gives exposure to those too intimidated (ie. me) to contribute to the random projects you can see on GitHub and the like. This is passion.

So Ive decided to take the plunge. Once things quieten down a bit of work and I have some spare time again Ill look into contributing into some of the Code52 projects, for no other reason than to learn how other developers work, and see some slightly more varied codebases as Ive been with my current employer for almost 5 years now, with the same codebase (plus a few bits and pieces) so it will be very interesting to see other perspectives and other thought processes in action.

Night all 🙂

BizTalk 2010 Upgrade From BizTalk 2006R2

Our client recently decided to upgrade their BizTalk instances from 2006R2 to 2010, and as we support several BizTalk applications it fell to myself and a coworker to upgrade them. I thought I would share my thoughts and experiences with this process…

Overall the upgrade has gone a lot smoother than I would have thought as BizTalk does have a tendency to be… temperamental. Although to be fair this is more than likely inexperience on my part than any serious fault of BizTalk (although there is one thing which is a Bad Thing, which Ill get to later). I find that (like anything I suppose) once you get your head around BizTalk’s ‘mentality’, troubleshooting issues is a straightforward process. Ok, so lets get into the meat and potatoes of the post :).

Global Assembly Cache

So, the first thing to note when upgrading a 2006R2 solution to 2010 is the GAC is now completely separate for .Net 4.0 DLLs. Yes, that’s right the ‘Global’ Assembly Cache is now two different folders:
– C:\Windows\Assembly [Anything pre .Net 4.0]
– C:\Windows\Microsoft.Net\assembly [.Net 4.0 DLLs ONLY].

So basically, if you have any legacy DLLs floating around, you will have to manage these in the old GAC and the resource DLLs generated as a result of the BizTalk orchestrations/maps/schemas in the new GAC. This is the ‘Bad Thing’ I was referring to earlier, as I find it quite annoying to have to switch between GACs when I’m deploying an application. While I don’t doubt there are valid technical reasons for the split part of me really wishes this could have been handled differently. Ah well.

Related to this, we now have two separate GacUtil executables, one for each GAC. These are located at:
– C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0a\Bin\GacUtil.exe [this is the .net 2.0 GacUtil]
– C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0a\Bin\NETFX 4.0 Tools\GacUtil.exe [this is the .net 4.0 GacUtil].

The GacUtil reference can found here for the .Net 2.0 executable. And the .Net 4.0 version can be found here.

Third Party DLLs and Resources

This is something that took a bit of effort to resolve. When you have third-party DLLs being referenced in your orchestrations/schemas BizTalk is very specific about how it wants these DLLs deployed. In our case, we needed to register the DLLs in the GAC AND add the DLL as a resource to the application. This is a bit tricky if you are trying to do a first time deployment from Visual Studio (ie. It wont let you deploy the application). So after much stuffing around I decided to write a batch file that handles all of these actions beforehand, after all automation is king 😉 . Once I have a spare few hours I will convert these to PowerShell… mainly because I love PowerShell! But in any case, the end result is that before someone goes to deploy one of our orchestrations, they run the batch file. This batch file will:
– Register the DLL in the relevant GAC.
– Creates a new Application on the BizTalk Management Server DB (more on this later).
– Adds the DLL as a resource to the newly created application.

And hey presto! Visual Studio will deploy your BizTalk application nicely.

BTSTask All The Things!

BTSTask is an executable found in the BizTalk installation directory. This handy little tool can do various things such as:
– Create a new Application
– Add a resource to an Application
– Import/Export Bindings
This was used in our pre-deployment scripts to get BizTalk into a state that Visual Studio doesnt complain. It would be nice if there was a way to do this sort of thing from Visual Studio without having to invoke command line utilities, but at least it gives us something new to learn! Speaking of which, the command line reference for BTSTask can be found here.

So thats the end for tonight. This was just a general overview type post, showing the main three issues/considerations I needed to be aware of when upgrading BizTalk 2006R2 solutions to BizTalk 2010. I will follow up in the next few days with the ‘nitty-gritty’ details, such as considerations for how BizTalk handles the AssemblyInfo files when upgrading the soultion (hint: it doesnt). So thats something to look forward to 🙂

Good night!

April Fools Humour

I was in a discussion on twitter last night about the ‘joys’ of visual studio, and how frustrating it was with (what should be) basic functions crashing in a great big ball of fire. This guy responded with a comment in a similar format to a commandment:

Properties are set in stone. Thou Shal not rename, nor shall thou covet the name of another property

This cracked me up laughing, so I challenged him to come up with a complete set. And here they are for all your April Fools pleasure!

1 – I am Struct, the class and the instance. I have freed you from the bondage of x86 assembler. You shall have no other assemblers b4 me.

2 – You shall not write your own compiler, or disassembler, or interpreter. For the CLR is all you need, If you try 2 map it you will go mad

3 – Thou shall not blame me for your errors of logic. Nor will you blame the build system. M$ Hath given you all that you need to deliver.

4 – Remember the Beer O’Clock. For labouring over code forth to long will cause you to overlook your own stupidity.

5 – Honor your team lead. For it is his ass that will beith on the line when your “I had a bad day” code gets into the release.

6 – You shall not murder. Except where a person usith an abbreviated type in the nameth of a variable. ie bIsItNew. Or usith an empty catch

7 – Thou shall not combine multiple frameworks that perform the same function. Nor multiple versions of the same framework.

8 – Tho shall steal, lots, where tho can. If one didn’t want there code stolen they wouldn’t have blogged it.

9 – Tho shall not use Exceptions as program flow control. Nor shall tho assume success unless it has been proven. For these are false.

10 – Properties are set in stone. Thou shall not rename, nor shall thou covet the name of another property.

These are straight from the tweets themselves, and I hope they give you many LOLs, as they certainly did for me 🙂 Happy April Fools everyone!

Effective Support

I had an awesome day today.

Not because of any new technology, or interesting piece of side work (though those are fun), but because today I was asked to fill in for the daily support tasks at work (our main contract is a support contract, with all of the investigations, bug fixes, data fixes and enhancements that come with it). You see, over the past 4 months for the most part Ive been on project work so it was a bloody nice breath of fresh air to get into the support queue (which we call Pending Technical 😉 ), and really hammer it down and get things done.

So as I was churning through the queue, relatively quickly, it got me thinking about support work in general and the various ways you can be of service to your client, while at the same time learning new things and staying interested in the work you do. Which brings me to the inspiration of this post… Effective Support.

I’m starting to realize, 6 years in, that support work is a really good grounding for developers, in various ways…

The first of these ways is troubleshooting; there’s no point being a fancy-pants developer, writing reams of code if you cant troubleshoot issues when (and they will!) start to appear. I find the grounding Ive had in troubleshooting over the past 5 years (how long Ive been working for my current employer, and my current client) has really enabled me to think on my feet, diagnose issues quickly (and usually correctly), and resolve them. Or if not resolve, at least put forward to be fixed. The Client wants results, even when the results may not be what they want to hear, and they would still much rather know whats going on so they (or you, as the case may be) can get it fixed as soon as possible. Not only that, they’ll appreciate the speedy response. Take too long, too frequently, and the Client will start to develop an apathy towards your relevant ‘Pending Tech’ queue, with little or no expectations. This is a Bad Thing… When the Client stops caring, they start looking somewhere else.

The second of these is communication; in the support world communication is everything. Whether it be written or verbal, communication will make or break a support ticket. Being clear in what you require from the Client will go a long way towards resolving your current task. If you get a ticket that makes no sense, like vague issue descriptions for instance, pick up the phone and call them directly. Don’t be afraid to talk to the Client… Ive found a single conversation will often resolve a multitude of issues, as you get instant feedback as to whats going on. Even more important than picking up the phone… See your Client. Most companies that outsource will have some sort of hot-desk arrangement, so talk with your manager and try and book at least half a day at the Client site per week. While there will be times where its simply not possible, the Client will always appreciate seeing the people on the other end of the line. If you stay stuck in a room with 4 walls, with no contact to the outside world, the support tasks you are responsible for will suffer, and even worse yourrelationship with the Client will suffer. Great communication with your Client leads to a great relationship with your Client… Which means you get to keep your job 🙂

Thirdly is maintainable code; again, using the fancy-pants developer example, there’s no point being so wonderful and awesome if the next guy that comes in doesn’t at least have a basic understanding of whats going on. While there are always going to be some areas that will be more complex than others, your code should be clear, concise and (I cant emphasize this enough), clearly commented. Now, I’m not saying that you should comment EVERY SINGLE LINE OF CODE IN EXISTENCE!!! (Imagine that shouted in a deep, echoed voice), but I am saying for areas that aren’t immediately clear, or require a deeper level of understanding, comments should be included to help the next developer in his/her mission to understand what you have done. I find, having been on the short end of the stick a few times, writing maintainable code from the start will go a long way to remaining effective and agile for your Client, otherwise your estimates will be bigger due to having to deal with the mess of code left from before. Lower estimates mean less money, which makes the Client happy, which means you get to keep your job 🙂

Lastly (for tonight at least) we come to business knowledge. Or ‘Domain Knowledge’ if you prefer. While we, as developers, will never have quite the understanding the Client does of their business (and Ive noticed we always tend to have a technically skewed view of things), it should be our job (especially as support developers) to increase our knowledge of the domain we are working in. Get use cases, functional requirements, policy documents, ANYTHING really you can get your hands on, and try to increase your level of contextual understanding of the business (how/where your application is actually used, and how the users use it). I think that business knowledge, in combination with communication can really enable your team to excel in the support environment, and are the most important points of what I’m trying to get across with this post. The Client always respects those who actually care about the business they represent, and who make the effort to truly understand what it is they are supporting. And with that respect, you get to keep your job 🙂

I hope this post has enlightened you, the reader, in what I believe to be three key points in giving Effective Support to your Client. Feel free to disagree/agree/modify my points in the comments, discussion is encouraged 🙂

Hobbies

Hobbies are awesome, and in my particular line of work more often than not our hobby is also our career, or if not this, then something computer related (personally, Im an avid gamer).

Im still not sure if this is a good thing, as while it does give us a passion for our work, and a drive to investigate deeper and further, sometimes things can go out of balance. For instance, you spend 8 hours of your day in front of a computer for work. You then get home and spend several more hours (depending on the level of your passion) investigating/reading/mucking around with other areas (or gaming). So you can get to a point where you spend about 15 hours of your day in front of a computer. This brings to the fore all sorts of medical issues over time; RSI, back pain, eye strain and headaches.

The reason for this post is to ask you this question: When was the last time you actually spent time doing something not computer related? Recently, Ive been making an effort for a better balance in my life for my computer/non-computer activities, and oddly enough Im enjoying it. I feel happier overall, and to be honest a bit more switched on when I do work. Ive started reading (non-technical) books again, and Im even making an effort with the garden.

The problem with being a tech-inclined person, I think, is our tendency for our hobby to become an obsession (although I suppose the same can be said of any hobby or interest). There is a crazy amount of information out there, and our passion wants us to know it all. Im starting to realise not only is this not realistic, but unhealthy to boot. Theres nothing wrong with being passionate about what you do, but you need to be careful how far your passions can take you, and develop the ability to be able to pull back before you fall down the precipice of obsession.

Its not a fun place.

Automation Is Your Friend

I thought I would share an experience with you all…

Recently I had a deployment go out into production. At the time, the build went very smoothly; no issues with the deployments of the websites themselves, the db schema update and post update scripts went through without a hitch. Wonderful!

Or so I thought.

Came in the next morning (and isn’t this how it always happens?), and about 9am we started getting suspensions in our BizTalk orchestrations. ‘Access Denied’. Hooray for 401 errors! An awesome little birdie informed me of a very basic mistake and left me to figure it out. So I trawled through our staging folders, checking and double checking the folders and files. Then I saw the answer to this problem.

“Configs”.

Oh yes, Id forgotten to copy the relevant production configs (specifically these ones defined what service accounts had access to the web sites) into the staging code. The parent web.config was OK, but the inner ones within the web sites had the dev service accounts listed in the file. Silly me.

So, an emergency change (blurgh) later and we had the proper configs in production, and all was (relatively) well.

Once the dust had settled, I decided to fix things so we never, EVER had this sort of issue again. And so, this brings me to the title of this entry – Automation is your friend. I modified our deployment script to copy the configs to the staging folder, and THEN copy the code to the relevant production servers. Problem solved 🙂

This got me thinking about automation in general, and how amazingly wonderful it is. Repeatability is what we, as developers, should aspire to. Because repeatability equals successful tasks, which means less time troubleshooting random or unexpected issues. Just for fun, I thought Id list some of the automation tools we use:

  • CruiseControl.NET (Continuous Integration)
  • Powershell (IIS/BizTalk Deployments)
  • Batch Files (Bits and pieces that dont quite fit, although the more I use Powershell, the more I realise that batch files are reduntant in my eyes)
  • SQLCMD (For SQL Script execution)

We also recently started using SQL Stored Procedures to automate some of our basic scripting tasks, as our main application is very reference-data driven, so when we are creating new entities, the amount of SQL we need to script is dramatically cut down. This is the second most awesome thing Im proud of in terms of what we’ve introduced to make our lives easier.

The most awesome thing is Powershell. Words cannot describe how much I fricking love this tool, and the power it gives you over your PC. While we are currently only using this for IIS and BizTalk deployments, the utter potential of this scripting language, where EVERYTHING is an object (this is where it has a HUGE advantage over normal batch file scripting) and can be accessed as such is mind blowing. I spent a good few days learning the basics before creating my first IIS deployment script, and it worked perfectly and (in my opinion) is alot more readable than batch files (its a weird thing to say I know, but hey).

Its amazing how many developers dont use automation in their environments. The ability to quickly (but sometimes painfully, depending on which CI tool you use) create a preview environment with a copy & paste job is amazing, and adds value to the client like you wouldnt believe. The ability to be dynamic and reactive to a client’s needs is essential in this day and age, and automation is a huge step forward towards those wonderful ideals.

CruiseControl.NET is our CI tool of choice, and to be honest Im not a big fan of it. Its completely XML driven (which can be a blessing or a curse depending on your taste), which makes it a bloody pain in the ass to troubleshoot sometimes. Saying that though, I do have a pretty decent handle on it now. But thats mainly a result of blood, sweat and tears trying to figure these issues out, as the log files arent always helpful. Ive recently introduced to a tool called psake. Psake is a powershell driven (yay!) scripting tool to create build scripts, without the messy XML. The syntax is clean and straightforward, and after my previous weekend of troubleshooting a rogue msbuild task (and trawling through rather large XML error logs), Im seriously considering the switch. This, in combination with a CI server called TeamCity (by the JetBrains mob) is my next personal project of investigation to see if its appropriate for our team.

So next time you have a tedious, repetitive task that needs to get done consider automating it, as which ever automation tools you choose to use its definitely a step forward from managing (or attempting to manage) everything yourself, manually.

Night all 🙂

%d bloggers like this: