Hope it doesn’t add up to a big issue!!
2 Likes
Post Mortem:
The upgrade went fine we had some issues outlined below but overall it upgraded cleanly and worked ok. There are some issues introduced by the Ice Common DB that you should be aware of and there still an outstanding issue with ECM Workflow that stopped working.
In this case we upgraded the following:
Kinetic 2023.1.10 → Kinetic 2024.1.16
Quick Ship 2021 → Quick Ship 2024.2
EPX CR 1.0.1082.1101 → EPX CR 1.0.1091.1097
MSSQL 15.0.4280.7 → MSSQL 15.0.4405.4
Total Time: ~35 hours from down time to up time
Number of AppServers: 13
Number of Db Servers: 1
Staff: 3
Issues Encountered
- Ice Common DB will not get created automatically if you are just doing an upgrade from version to version. You have to go into Admin Console and create / add a new Demo Db which will then create an Ice Common DB.
- If you already have an Ice Common Db and re-deploy an AppServer Epicor will sometimes create an additional Ice Common Db named the same as the prior one but with the name duplicated ICECommon_4_3_100 → ICECommon_4_3_100_4_3_100. This makes everything super fun and has already been reporting as a PRB and it is getting fixed. IF this happens then some environments will be looking for the new " fun name " and others will be looking for the old regular name.
2.1 The fix for us was to remove the (read only) attribute from the Db and rename it to what was needed during each deployment back and fourth - Ice Common Db is included now in 2 Views within the main Epicor Database Ice.FieldHelp and Ice.ReportStore. If you are doing full DB replication your replication will blow to pieces you need to explicitly remove these 2 views from the replicated Articles and take another fresh snapshot for that to work. Not that big a deal except snapshot takes sometimes hours to process locking up the main Db while it happens.
- .NET 4.8 is required (period) on the Server and the Workstations. They claim not needed on the server but if you ever open the client on the server for anything related to BPMs or Functions you’ll need it. If you are in Windows 2019 4.8 Runtime comes installed already, you need the 4.8 SDK which has a known issue where it can’t be installed using the regular installer. You’ll have to install it using Visual Studio 2022 BuildTools . Symptoms are that none of your Functions or BPMs will compile the App Freezes and or crashes with a Mono DLL error
- Epicor moved a documented endpoint from /v2/Environment to /Environment but none of the documentation is updated. This Endpoint lets you get a list of Companies and or Plants that a user has access to without needing to first give it a company id (chicken / egg).
- ECM will not bring over attachments from the AP Invoice Workflow see: Upgrade and now getting cryptic error in AP invoice workflow as far as we can tell the error isn’t in Epicor it is causing an issue in ECM we are trying to troubleshoot and have opened a ticket.
- The application will intermittently have a session timeout and boot you out in the middle of working this happened quite a few times at the beginning while we were deploying things, it seems to have stabilized now
- Quick Ship changed the way that UPS is authenticated so this had to be re-done for our carrier.
Many many thanks @Olga who came to our rescue in the middle of all this and got us across a pretty big issue see #5 above we rely on this endpoint for all our customer and sales portals.
This is the first time we are deploying the Kinetic HomePage to everyone, so far it hasn’t been an issue. We were able to create layouts for all the major functions ahead of time and assigned them to each user via DMT (that was nice)
20 Likes
Finally were able to replicate #7 above, case opened PRB a coming
Demo Below: 500+ Angry user’s with pitchforks yesterday. 
7 Likes
We noticed this during our 2024.1 go live as well across our 3 app servers, but after the initial setup we don’t do changes in prod enough to encounter it “in the wild” yet. I will attach myself to your PRB as well.
1 Like
Interested in that architecture is that 13 app servers for Production or 13 app servers across the board and if 13 app servers for production are you running multiple appserver (application pools/website) on the same host?
Eg.
HOST1 ---- ProdAppServer01
|
---- ProdAppServer02
HOST2 ----Prod AppServer03
|
---- ProdAppServer04
???
3 physical servers for production one app server each in one load balancer
We have 13 app servers across the organization for development training pilot and testing (and 13 databases)
6 Likes
Do you use HA?
Of course. I crack jokes all day. 
3 Likes
Sql Enterprise?
SQL Standard is a beast by itself with proper resources and setup.
No need for Enterprise at this level.
3 Likes
The ONLY reason one might look at Enterprise is if you think you might be in the cloud someday (in your own instance), then the license is transferable saving some money.
1 Like
You do if you want HA with Read only Replicas to take advantage of the BAQ offload
2 Likes
Only one Task agent? What settings, just the default? Sorry if getting a bit too in depth @Haso do you recall what you had used in past.
I was curious about the Max Concurrent tasks setting value. What value do you have that set at?
We have 2 Agents running on different servers.
2 Likes
Max Concurrent Tasks setting value??
1 Like
Whatever is default
1 Like
That would be 20… Thanks for all the clarification
1 Like
@josecgomez do you have to run the whole thing? I notice the Ice Common DB gets created in task 200:
