Archiving - What have you all used that are longer term EPICOR users? I have been using EPICOR since 2006. I have a lot of older POs, AR, AP records that are causing users time to sort thru when searching looking things up. What do you all do that are long time customers with literally many years of data for this? Any good archive solutions out there? I have reached out to EPICOR their response is there is no epicor archive process/Tool/Module. Thanks folks for this vital info.
1 Like
We’ve only been live about 18mo but this will come into play eventually for us.
Hopefully the “solution” is NOT to continually beef up servers so they can burn through gazillions of records on a search/lookup.
I’ve always thought it would be neat to do a fresh install when the system starts to meet its limitations with the # of records.
Archive the old db, spin up the fresh one, load in all your master data/config via dmt, import your UBCs and good to go! Probably way harder than it sounds tho 
Epicor has a built in Purge and Summarize feature that can be run once in a while (I’ve never tried it)
5 Likes
Will look into it…thanks for the tip.
I will be asking EPicor dbase tech person early next week in meeting with them on that feature Jose, so thanks for mentioning that. I will keep you all posted on the results of that meeting too. For my info Jose how long has your company been using EPICOR how many years history do you have?
1 Like
We went live in 2018 we have a ~300GB Db right now and would love to summarize some stuff.
2 Likes
We have our really old ERP system (Qantel) in an Access database that only has Sales and Financial information in it. Our old Epicor 9 Progress database is still running on a VM on one of our servers for archival information. I am also curious how others are saving off information that Sales, Engineering and Accounting want us to save forever? Is there a better platform to put all this data into so if someone needs archival information they have access to it without keeping outdated operating systems running?
Been on Epicor since 1997
Just ran a test BAQ counting parttran records.
read 2.3 million records in 10 seconds.
I’m on Epicor Public Cloud.
If your using table index’s you should not have performance issues.
SQL is build for this stuff, let the big dog eat.
5 Likes
It’s nice to know there are other hoarders out there … ours is coming up on 500GB (live in 2012)
Archiving comes up a lot in conversation … but it never gets past “We need to figure out a good way to do it”
Very interested to hear of working solutions
I would suggest purging change log records to reduce you DB size as the first step before you purge actual data.
3 Likes
And maybe while your at it, review what your change logs are actually logging.
As a rule, chose specific fields, not all of the damn things.
1 Like
I tried this once in a dev database years ago. “Tried” is generous. I don’t think I let it finish as I needed the DB to actually work in.
I think it would literally take days for it to do its work. I don’t know when you could ever schedule that.
But I love this question - I hope I am wrong and there is a good way of doing this. I am curious also. Our DB is 250-300 GB.
To be fair though, the 250-500 GB sizes mentioned are in the range where a skilled DBA becomes a requirement to maintain functionality. More so if the data isn’t normalized! SQL can certainly handle it as long as someone’s at the wheel.
For SaaS users we’re stuck with the DBA we get. I’ve been exporting historical data outside Epicor, targeted by what users need rather than trying to support everything. We have some externally related pre-Epicor or adjacent stuff that needs to be joined, so that works out nicely. Seems to go a lot quicker for users to find an order or invoice by its metadata outside Epicor, then have an order or invoice number to navigate to directly, but then others have a lot more data so YMMV.
1 Like
Screencap of the prompt screen in case anyone’s interested. No, I didn’t click the Gears button in the middle of a Friday…though I was TEMPTED
1 Like
It seems to me the purge feature was put in when they launched E10 and was really designed to be run only during upgrades. I think the comments I remember from back then pertained to those users who had been on Epicor for quite some time by that point and having a decade+ of records that would need to be uplifted from Progress to SQL. I don’t think it was something that was intended to be run as a normal thing.
We’ve been on Epicor for 15 years now and have a 330GB database.
1 Like
Found my notes. I tried it in 2022.
I tried 4.3 years’ worth of transactions. I cancelled it after 17 hours, where it was approximately 10% through the PartTran records.
So that extrapolates to a literal week of processing.
You’d have to do like 3 months at a time, every weekend or something, for months till it was done.
EDIT:
Here’s the screenshot I took then.
- Again, I cancelled the process well before it finished. Fair warning.
- Cutoff was 3/1/21
- This is a VERY common part here - we have transacted it basically every day since December 2016
- You can see some of the summaries - they are all dated 3/1/21
- Most of the original transactions are gone, but there some oddballs that remain from June 2020.
But I’d love to do or see this done right or to completion.
1 Like
Your giving me flashbacks of the fun progress dump and load days.
We had a 1TB database with data stretching back to 2009. Lots of mistakes, process changes, companies coming and going, the old E9 UD fields on every table. It was taking its toll on performance and infrastructure so we took the nuclear option of building a new database. The purge functions don’t cover enough areas. I have created an idea ( KIN-I-4254) to add more items such as shipments and invoices.
But yeah we survived a database rebuild. It wasn’t pretty. I never want to do it again. The biggest ongoing issue is with RMA’s, cases and credits and not being able to reference back. Oh and Docstar is broken and Epicor cant fix it lol
But yeah, Epicor need to cover the entire product with purge functions.
1 Like
Am I reading this right, you have been on Epicor since 1997 and only have 2.3 million rows in your PartTran table?
Have you done purges on it or is this a subset?
We’ve been Live since 2009 and have 11,250,000 rows in PartTran so I would expect you to have far more? Not trying to derail the thread, I just didn’t see a way i could PM you.