Last week and again this week we’re receiving reports of Server Side errors, where the Inner Exception seems to imply DNS lookups are failing. We’d like to dig a bit more into exactly what’s going on, so ideally we can isolate the cause so we can fix it. To this end I dug into Windows event viewer on our Epicor server but couldn’t find any entries regarding a DNS lookup failure.
Any suggestions? Does this error mean what I think(DNS lookups are failing)?
It’s being thrown at a custom code block, so something a developer wrote is not working.
Can you share the code?
You’ll probably want to make sure the remote name in your code is added as a entry in your host file on that machine.
@Aaron_Moreng last week when the error first came up I thought about adding the IP in the hosts file(quick 5 min fix), but that seemed like a big band aid - implies DNS is unreliable. Or rather if DNS is unreliable lets figure out why and fix DNS rather than putting a bandaid on the resulting symptom. Although the odd thing is if DNS was unreliable I’d expect to be seeing a lot more strange problems, but we’re not. The only evidence I know of in regards to DNS being unreliable is on our Epicor server(e.g. this error).
I don’t see the business object and method in the error message(if it’s there and I’m missing it can you point me to it?). Based on the screenshots from the end user my guess is this is a preprocess BPM on a method related to customer shipments(was in customer shipment screen), so my best guess is the code is below…
Ice.Common.BusinessObjectMessageType XX = Ice.Common.BusinessObjectMessageType.Information;
Ice.Bpm.InfoMessageDisplayMode YY = Ice.Bpm.InfoMessageDisplayMode.Individual;
int i = 0;
foreach (var ttShipHead_iterator in (from ttShipHead_Row in ttShipHead where ttShipHead_Row.ReadyToInvoice select ttShipHead_Row.ReadyToInvoice) ) {
i++;
}
if (i > 0) {
//PublishInfoMessage("Already closed",XX,YY,"","");
return;
}
string serverURL = $"http://eureka.ts-local.net:8080/bpm/custship1/{packNum}/1";
var client = new System.Net.WebClient();
Stream data = client.OpenRead (serverURL);
StreamReader reader = new StreamReader (data);
string str = reader.ReadToEnd ();
data.Close ();
reader.Close ();
string [] ret = str.Split('~');
if (ret[0].Equals("Ok")) {
} else if (ret[0].Equals("Warning")) {
PublishInfoMessage("===== WARNING =====\n"+ret[1],XX,YY,"","");
} else if (ret[0].Equals("Error")) {
PublishInfoMessage("===== WARNING =====\n"+ret[1],XX,YY,"","");
// CallContext.Current.ExceptionManager.AddBLException("===ERROR===\n"+ret[1]);
}This is an odd way to do this…do you have a custom API hosted on that server? Is there a reason you aren’t calling the Epicor BO directly, and instead using this custom thing?
i would still think having that host added is a prudent move, since you are indeed calling it by name.
You probably saw this, but might want to look at internal connectivity to that server
c# - System.Net.WebException: The remote name could not be resolved: - Stack Overflow
I don’t know if this is on the same server your running Epicor on, but if it had a momentary lapse in connectivity, it could possibly throw a DNS error
You are making a web call in your BPM… what does it do? 
Its returning a tilde delimited string???
@Aaron_Moreng When developing with in the Epicor “ecosystem” we are constrained by the tools / solutions Epicor provides. Which in my personal opinion open source tools tend to be better. As an example look at the strategic direction Microsoft is going, they are embracing open source tools(give developers choices to pick the best tool for the job) and moving away from the walled garden approach… So the thinking for this design approach is to create external web services(APIs) as that allows us to use the best tool for the job 
I think I’ll take your advice and just add the hostname to the hosts file, as this DNS lookup issue seems to be specific to our Epicor server(makes me question if this is a DNS server issue, or some kind of a Windows configuration problem with DNS)…
Out of curiosity what does this call to the API do? (in this case) if you don’t mind sharing and what other tool does it use?
@josecgomez I’ll have to ask what the code is doing(I didn’t write it). More concretely when I say “use the best tool for the job” the options with in Epicor are C# and widgets (as far as I know and I’ve been told we really shouldn’t use C#, we should stick to Widgets by Epicor professional services). By using external web services we can use what ever language we believe is the best tool for the job, in this case we used node.js. This approach also allows the developer to use the editor they are most productive in(vs the 1980s era editor Epicor offers), and it allows us to check our code into Git(and all the power that comes with Git). Although we’re not currently doing much in the way of devops best practices, this approach also opens up this as a possibliity…
Having said all that, we would love to hear any criticisms or concerns with this approach as it’s relatively new for us… If helpful we contracted with a large 3rd party Epicor developer and this was the approach they used, that is this wasn’t our idea, we stole it
I understand that but you are shoving an “Ever Given” size boat in the middle of your Epicor business objects by having to go through the http stack for something as trivial as checking if they are shipping enough parts or if that customers approved etc.
I guess I’d have to understand the advantages beyond I don’t know C#, right tool for the right job. Epicor’s BPMs back end uses C# and that is by definition that right tool for the job anything else that requires you to go outside the current flow ads a HUGE amount of overhead not to mention error prone issues like your DNS not working.
Again it depends heavily on what the BPM is doing, but if that data is available within the ERP going out to an HTTP Stack to make a call just so that you can use node seems insane from a performance, scalability and reliability side of things.
Puts on Professor Hat:
A Language is just Syntax that can be googled or read in a manual, same goes for a tool, what is important is to have the ability to understand the problem and how to solve it, then chose the technology that better serves your purpose from a technical, performance and reliability stand point. I teach all my students that choosing the language and framework you know just because you know it, is not the appropriate approach, what you gain in ability to develop code quickly you lose in maintainability, performance and reliability.
One should never for example use Node for an embedded real time system, or COBOL for a website… can you do it? Sure… but come on! Right tool for the right job so although I appreciate your conundrum I’d argue that staying close to the native tools is the best approach.
Hat Off
If this is their approach I would stay far far far away from them.
My 2 cents is that it’s nuts to use 20+ lines of code to call a dedicated webservice to do what you can do with probably 3 lines of code in C# in the BPM. I’m a free agent now, so I’m for hire if needed (whether for code correction or educational purposes…teach a man to fish)
@josecgomez thanks for your feedback, it’s really appreciated.
In my opinion DNS should be rock solid, it’s a core service most technology we use depends upon. If we have unreliable DNS we have much bigger problems outside of just Epicor(e.g. we would shutdown our production lines
I like your point about performance and adding a huge amount of overhead - this has me thinking… Great discussion!
We’ve been fighting concerns that Epicor is slow, so we most definitely don’t want to make things worse… In my experiences HTTP GET of a local web service is relatively fast. I didn’t measure this specific endpoint but I would expect 10s of milliseconds up to a worst case 100s of milliseconds for the call to complete(SOAP APIs in E9 took hundreds of ms and that seemed really slow to me). If that is indeed the case the overhead is a fraction of a second - I would consider in the noise (from an end user perspective).
So our thinking was if we can dramatically improve our tooling(e.g. we can use Git to know who changed what, when, rollback changes, etc) and we can significantly improved developer productivity with out hurting performance from a human perspective(e.g. a person generally wouldn’t notice if it takes 3 seconds or 3.2 seconds for customer entry to open - either way they complain Epicor is slow because the screen takes 3 seconds to open :). Keep in mind I’m talking from a 50k foot view, I don’t know the specifics of this endpoint(what it’s doing). I see your point in that the assumption seems to be we’re doing something really really simple in that endpoint, and if that is the case then I agree just do the simple “thing” in C#. Long story short it seems more like a tradeoff as apposed to a right and a wrong way to do it, but I’ll think more about this. You’ve got tons more experience in Epicor development, so I will continue to think and try to poke holes in my thinking in case I’m missing something.
Thanks again for your feeback
Bringing up an HTTP Stack is incredibly expensive you have to setup and tear down HTTP / TCP it may seem like its small but compared to doing a local query already on the server you are taking 10s or 100s orders of magnitude. And that’s just for brining up the stack and establishing trust and communication, then you have to add the actual logic on top of it if then from node you are reaching across to Epicor to get some data say via REST then you can double triple and quadruple the overhead.
Further more if your node instance were to go down you’ve now effectivly completely broken your ERP system lots of food for thought on this but I’d be hard pressed to be convinced that this approach should be considered except in extreme situations where you MUST check across the network for something and even that should be handled async.
And pardon my arrogance (I’m a little bit passionate if you can’t tell) but I’d say its definitely a right and wrong as in doing it this way is wrong, it fundamentally will break your ERP and cause huge headaches down the road and the benefit is very very small. I’d beg ya to re-consider and I’ll leave it at that.
@josecgomez thanks again. Really good feedback and food for thought!
Healthy open discussions like this are a great thing, thanks for all your input
I’ll hardcode the IP of our webservices server in our hosts file on our Epicor server, unless anyone has troubleshooting advice as to why Epicor seems to fail looking up this host but we don’t have any evidence of any other DNS lookup failures(I couldn’t find anything in the Windows logs providing more detail).
I guess you could use External BPMs, I know @Carson does this all the time.
I usually use the C# Code Block 80% of the time, thus far no issues 
on-prem.
