Bulk image import? (50,000+ part thumbnails)

Next task: To extract png files from our Solidworks PDM for 50k+ prts, and bulk import them into Epicor so we can include them on purchase orders when we get cutting and folding done. As well as to just make things look pretty :slight_smile:

Iā€™m working on the first part, getting the image, and assuming I get that far, I think I can use image management in epicor to bulk import the images:
image

Any idea what Import Image Category/Subcategory from File does?

Iā€™m then thinking I just need to DMT Update to set the Image ID, but then there is a path to a jpg there also, assume this is deprecated or to be avoided?

Any experience anyoneā€™s had attempting something similar is MUCH appreciated. I think we will be looking at using CADLink in future, but thatā€™s a year or two away.

I wonder how many it can handle at a time without choking.

What size are the images?

Well, weā€™ll start out small I guess and build upwards. I dont know yet what size they are yet til i get the extraction going, but guessing 128x128, or maybe 256x256.

I have done this before, will check my notes, but not at that scale.

Iā€™ve kept part and employee images in the database. Honestly, SQL Server is a very expensive place to store images/blobs.

Personally, Iā€™m looking at moving images to cheaper blob storage. Having images available as web resources makes them easier to use in multiple places: SSRS reports, Kinetic Screens, mobile apps, etc. They will cache better as web resources too, especially for large scale projects

Just a thoughtā€¦

3 Likes

Thanks Mark, SSRS reports is the primary requirement for accessing these thumbnails for now. I havenā€™t gotten into report design yet, had left that with our consultants, but had assumed the images would need to be in the database to get pulled out. (We are coming up 10 months into this epicor project, and some months away from go-live)

What do you mean by ā€œexpensiveā€ though? At this point I was seeing the SQL server as a magic bottomless pit for all our production data (we are on-prem also). Our situation is exacerbated by the fact we have two companies within the same instance sharing the same Global parts, and its looking like I need to load the images twice, in each company. so 50k becomes 100kā€¦ its certainly tweaking my ā€œinelegant sensorsā€

Iā€™d love to know more about to to make them available as web resources and if SSRS can pull them in that way; anyway some learning to be done.

A couple of early tests: 310 large jpg images, turned out they were 4k resolution, about 2Mb each, definitely choked with an out of resources error after some minutes of appearing to have hung. Resized then all to 150x100 (Powertoys image resizer) and 2kb, instant success but a little chunky to look at, redid the resize at ā€œiPhone resā€ 320x568, 4kb each still pretty instant and looking good. Wonā€™t be the final resolution but looked promising

Have you thought about storing the actual files instead of blobs?

Yes; I can certainly shove them all in a fileshare folder or folder structure (OK not yet, to be precise thatā€™s next weekā€™s job, figuring out how to extract them from Solidworks PDM)ā€¦ but will I be able to pull them from there into SSRS reports ā€œeasilyā€?

Shove them in a file share named by {partnum}.jpg

Then drop that share as a virtual directory on IIS

And on your Ssrs report just add an image with the URL http://your server/VirtualDirectory/{partnum}.jpg

Works like a charm and it couldnā€™t be easier

7 Likes

Thanks Jose, that sounds almost too good and simple to be true! Iā€™ll hold fire on the epicor image import and pursue this approach.

Makes it easier for us to maintain also, if Design changes a part we just need to update the thumbnail and drop it into the folder, which I think PDM can automate for us. :crossed_fingers:

(Weā€™ll go with .png as I think its better for vector renderings.)

2 Likes

Iā€™m interested to see how you go as we have a similar setup. Solidworks/ PDM (workgroup) / Cadlink / 170k parts. Currently the part images are pushed into Epicor db via Cadlink which means only parts processed with Cadlink have images. The images are available via a URL from a custom web app we have for time entry. This is useful for reports etc so I would recommend that.

In terms of storing image files may I suggest /images/parts/{partNum} so you can add /images/employees/{empId} later

Brett

3 Likes

How has Cadlink worked out for you? We are looking at it down the track as phase 2 of our Epicor implementation. Phase 1 is harder than expectedā€¦

CADlink has been ok. It is not a polished app eg if you start it without a running Solidworks session it gives a null reference error that is not at all helpful.
On the plus side you can get it customised for a reasonable cost and that way you can get it to do exactly what you want. That is how we got the images imported.

Brett

@Chrisw Sorry about the late reply, but looks like @josecgomez nailed it there.

As far as the image category from file, the help mentions if you have previously exported images with the image category and subcategory, then you can reimport it. It doesnā€™t give any format. But I suggest you could just do one image export it and then take a look at the category and subcategory. I have never needed to use subcategories.

Use the part DMT is to populate the imageid field to store the imageid of the picture, Iā€™m sure you have worked that out already.

Earlier versions (E9, if I recall correctly) you only didnā€™t have the ability to store in the Db and you put in a file and path. Interestingly the DMT still has the ImageFileName field. There is some mention of image settings in the company configuration, but it looks like is only around image category and subcategory. Nothing to do with the file system, I am guessing you could use it to store the URL to the image file locally (File system or URL) or in the cloud (Blob storage or S3 bucket) then use that in the RDD for the report.

When you all are using URLā€™s to images saved on a file share (or separate web server, like our ecommerce platform), how are you adding them to Epicor? We were hoping to use them on Part Tracker, for fulfillment, and EKW for shipping and receiving.

  • The Part > Attributes tab has the File button, but that search only shows images loaded via Image Maintenance (not DMT into Part Entry). I like the concept of not filling up our SQL database when itā€™s avoidable.
  • I tried setting the ImageFileName to the public URL from our webpage with DMT on Part, but Part Entry errors with an ā€˜Invalid Imageā€™ (URL has a .jpg extension).
    The DMT field notes say itā€™s ā€˜relative to images/prod_img directory on the web serverā€™. I didnā€™t see anything in Company Maintenance or Configuration where weā€™d set that. Or in the .sysconfig file eitherā€¦

image

Suggestions? Thanks!

We created a small web service that gets the images from a folder and are displaying them bia the website widget In the example below its doing a bit more, it is getting the thumbnail from Solidworks Pdm, and is accessed in the web widget via http:///thumbnails/ or similar, then caching it in a folder on the server for quick access next time. Buttons access the part in pdm.

Really annoys me having the wasted footer space below the website widget and for matter above it also, slightly mitigated by the ā€œSolidworksā€ title there but thatā€™s unnecessary anyway. Hopefully will figure out one day where to modify the css for this widget on the server to eliminate them.

1 Like

Yeah, they mean the Epicor Web Server.

Yes, a relatively simple BPM could be made to retrieve the image on:
ERP ā†’ BO ā†’ Image ā†’ GetRows

If you want to assign it from there, youā€™re in for a more moderate level of customization.

Frankly, I like what @Chrisw is doing. Why store images in a database? :thinking:

Iā€™m suggesting the same, just getting into the built in place.