ODBC Slow (Newbie Question)

Juliet,

Wow. I can run the example from my previous message in RB or Access in
less than five seconds (although our db is only 2.5GB, the server is
350MHz).

The general rule of thumb, as Todd noted, is to always join on indexes.
The corollary is to filter and sort on indexes, if possible.

But to give you an idea of why this is important, here's what a query does.

If you query LaborHed for EmployeeNum "XYZ" the query has to read the
whole dataset to find your records.

If you start with Company, and join to LaborHed on Company.Company, then
filter on Company.Company (not LaborHed.Company), your query dataset is
instantly reduced.

Further, if you then filter on any index (such as LaborHed.PayrollDate),
the query dataset is reduced further.

The objective is to reduce the dataset to the smallest possible size for
your query to search. The smaller the dataset, the quicker the query
runs.

If you filter on any non-indexed field, you're automatically increasing
the size of the initial dataset, because the Progress server will just
dump everything into your PC for your query to pick through. I imagine
that's what's happening with your query.

Also watch out for outer joins. If you start with Company and create an
inner join to LaborHed, then an outer join to EmpBasic, be sure that any
sorting or filtering is done in the first two tables, and not in EmpBasic.

And that brings up one final caveat: If you filter on a field that is
returned from an outer join, your outer join becomes an inner join, and
thus you "lose" data from your query. If you have to do this, the best
thing to do is dump to a static table and then do your filtering there.

have fun,
john



Juliet Martin wrote:
> Thanks. I have done all of the following except the LaborDTL table I don't need. Just laborhed and empbasic. Still slow. The same report runs in roughly 1 - 2 minutes in Report Builder (and I am not using Company on that one as the master table). I just ran this report with all the changes suggested and it has been going for 10 minutes now.
>
> Why such a performance drop?
>
> I will look this over again maybe I am missing something.
>
> Juliet
OK. I have written a simple report using EmpID and LaborDtl and used the select expert to filter down to just my employee code and today's date. I copied a report I had made in report builder. Same joins, etc. This thing has been accessing for over 10 minutes now.

What am I doing wrong now?

I wished they would have just kept report builder.

Juliet
It just came up. Labels but no data. Never mind. I guess I have more problems than just ODBC.

Juliet

-----Original Message-----
From: vantage@yahoogroups.com [mailto:vantage@yahoogroups.com]On Behalf
Of Juliet Martin
Sent: Tuesday, May 17, 2005 2:50 PM
To: vantage@yahoogroups.com
Subject: [Vantage] ODBC Slow (Newbie Question)


OK. I have written a simple report using EmpID and LaborDtl and used the select expert to filter down to just my employee code and today's date. I copied a report I had made in report builder. Same joins, etc. This thing has been accessing for over 10 minutes now.

What am I doing wrong now?

I wished they would have just kept report builder.

Juliet
Always make sure you have a select on the company maybe even try both tables

Frank Zeigafuse
Innovative Office Products
General Manager
Direct Phone: 610-559-6369
Email: fzeigafuse@...



-----Original Message-----
From: Juliet Martin [mailto:jmartin@...]
Sent: Tuesday, May 17, 2005 3:50 PM
To: vantage@yahoogroups.com
Subject: [Vantage] ODBC Slow (Newbie Question)


OK. I have written a simple report using EmpID and LaborDtl and used the
select expert to filter down to just my employee code and today's date. I
copied a report I had made in report builder. Same joins, etc. This thing
has been accessing for over 10 minutes now.

What am I doing wrong now?

I wished they would have just kept report builder.

Juliet



Useful links for the Yahoo!Groups Vantage Board are: ( Note: You must have
already linked your email address to a yahoo id to enable access. )
(1) To access the Files Section of our Yahoo!Group for Report Builder and
Crystal Reports and other 'goodies', please goto:
http://groups.yahoo.com/group/vantage/files/.
(2) To search through old msg's goto:
http://groups.yahoo.com/group/vantage/messages
(3) To view links to Vendors that provide Vantage services goto:
http://groups.yahoo.com/group/vantage/links
Yahoo! Groups Links
Thanks. I think that helped. Now if I can just get records to show up... I love learning. ;-)

-----Original Message-----
From: vantage@yahoogroups.com [mailto:vantage@yahoogroups.com]On Behalf
Of Frank Zeigafuse
Sent: Tuesday, May 17, 2005 3:37 PM
To: vantage@yahoogroups.com
Subject: RE: [Vantage] ODBC Slow (Newbie Question)


Always make sure you have a select on the company maybe even try both tables

Frank Zeigafuse
Innovative Office Products
General Manager
Direct Phone: 610-559-6369
Email: fzeigafuse@...



-----Original Message-----
From: Juliet Martin [mailto:jmartin@...]
Sent: Tuesday, May 17, 2005 3:50 PM
To: vantage@yahoogroups.com
Subject: [Vantage] ODBC Slow (Newbie Question)


OK. I have written a simple report using EmpID and LaborDtl and used the
select expert to filter down to just my employee code and today's date. I
copied a report I had made in report builder. Same joins, etc. This thing
has been accessing for over 10 minutes now.

What am I doing wrong now?

I wished they would have just kept report builder.

Juliet
The labordtl table is normally huge, at my last company it was hundreds of
thousands of records and anything I tried to do to it via odbc was slow.

Jeremy


[Non-text portions of this message have been removed]
Figured it out. Still in Report Builder mode. Don't put 's around text. I guess I get the Duh! Award for the day.

-----Original Message-----
From: vantage@yahoogroups.com [mailto:vantage@yahoogroups.com]On Behalf
Of Juliet Martin
Sent: Tuesday, May 17, 2005 4:16 PM
To: vantage@yahoogroups.com
Subject: RE: [Vantage] ODBC Slow (Newbie Question)


Thanks. I think that helped. Now if I can just get records to show up... I love learning. ;-)
I have noticed this even in Report Builder. Any time you use the LaborDtl Table add 5 - 10 minutes to your report print time. At this point I have most reports printing to AdobePDF. The printers time out anymore...

Juliet

-----Original Message-----
From: vantage@yahoogroups.com [mailto:vantage@yahoogroups.com]On Behalf
Of jeremyrleonard@...
Sent: Tuesday, May 17, 2005 4:16 PM
To: vantage@yahoogroups.com
Subject: Re: [Vantage] ODBC Slow (Newbie Question)


The labordtl table is normally huge, at my last company it was hundreds of
thousands of records and anything I tried to do to it via odbc was slow.

Jeremy
Juliet,

Here are the query rules to live by (for all queries - ODBC, RB, 4GL,
whatever):

#1:
Company is _always_ your master table

#2:
Filter and sort for your company on Company.Company
(even if you only have one company in your database)

#3:
Next join should be to a header table, even if you're not going to
use any data there

#4:
Final join is to your detail table.

So for you:

Master table is Company
Join Company.Company to LaborHed.Company
Join LaborHed.Company to LaborDtl.Company
and LaborHed.LaborHedSeq to LaborDtl.LaborHedSeq

First filter and first sort:
Company.Company

Second sort:
LaborHed.PayrollDate

Finally, filter on LaborHed.PayrollDate = Today and
LaborHed.EmployeeNum = You.

You've now sped up your query by at least 10X.

have fun,
john



Juliet Martin wrote:
> OK. I have written a simple report using EmpID and LaborDtl and
> used the select expert to filter down to just my employee code and
> today's date. I copied a report I had made in report builder.
> Same joins, etc. This thing has been accessing for over 10 minutes
> now.
>
> What am I doing wrong now?
>
> I wished they would have just kept report builder.
>
> Juliet
Thanks. I have done all of the following except the LaborDTL table I don't need. Just laborhed and empbasic. Still slow. The same report runs in roughly 1 - 2 minutes in Report Builder (and I am not using Company on that one as the master table). I just ran this report with all the changes suggested and it has been going for 10 minutes now.

Why such a performance drop?

I will look this over again maybe I am missing something.

Juliet

-----Original Message-----
From: vantage@yahoogroups.com [mailto:vantage@yahoogroups.com]On Behalf
Of John Sage
Sent: Tuesday, May 17, 2005 8:24 PM
To: vantage@yahoogroups.com
Subject: Re: [Vantage] ODBC Slow (Newbie Question)


Juliet,

Here are the query rules to live by (for all queries - ODBC, RB, 4GL,
whatever):

#1:
Company is _always_ your master table

#2:
Filter and sort for your company on Company.Company
(even if you only have one company in your database)

#3:
Next join should be to a header table, even if you're not going to
use any data there

#4:
Final join is to your detail table.

So for you:

Master table is Company
Join Company.Company to LaborHed.Company
Join LaborHed.Company to LaborDtl.Company
and LaborHed.LaborHedSeq to LaborDtl.LaborHedSeq

First filter and first sort:
Company.Company

Second sort:
LaborHed.PayrollDate

Finally, filter on LaborHed.PayrollDate = Today and
LaborHed.EmployeeNum = You.

You've now sped up your query by at least 10X.

have fun,
john
I can't imagine why this report is so slow on your systems if you have all
your joins correct. We have lots of ODBC reports on labor tables and they
range in speeds from instantaneous to maybe 30 seconds for reports with more
detail that span larger date ranges. Never has an odbc report taken several
minutes to run - and we do all of our reporting thru odbc and have for the
past year or so. If you want to try sending me your report
(sarah.vareschi@...) I will run it on our system to help you to
determine if the problem is with your report or something else (maybe odbc
config or network?).
Even with all the best techniques for linking and filtering I have labor reports that take far too long (in RB but same would apply to Crystal). One other thing I often do is to pick an index for the table and make sure the filtering follows everything in the index PLUS set a filter for some given transaction number (in the index - primary index is best) that represents the start of my "range"....if the report is just for data this year then once a year I reset the start pointer to the first transaction of the year. When a filter follows an index like this is goes straight to that record without reading and testing all the prior records and then processes from there. On a huge table with multiple years this can cut the time dramaticaly. Takes a little annual tweaking but the difference in performance in January is HUGE compared to December so I know it helps even for one year.
-Todd C.

PS - I am currently developing benchmarks on various scenarios of links and filters for a presentation at the now "in-planning" User-for-User meeting Calvin Dekker is setting up for VMUG in Chicago in August. In addtion to my own "tricks" I am browsing the past group posts and pulling in any other info I can on optimizing reporting links and DB navigation.

-----Original Message-----
From: Juliet Martin [mailto:jmartin@...]
Sent: Tuesday, May 17, 2005 4:22 PM
To: vantage@yahoogroups.com
Subject: RE: [Vantage] ODBC Slow (Newbie Question)


I have noticed this even in Report Builder. Any time you use the LaborDtl Table add 5 - 10 minutes to your report print time. At this point I have most reports printing to AdobePDF. The printers time out anymore...

Juliet

-----Original Message-----
From: vantage@yahoogroups.com [mailto:vantage@yahoogroups.com]On Behalf
Of jeremyrleonard@...
Sent: Tuesday, May 17, 2005 4:16 PM
To: vantage@yahoogroups.com
Subject: Re: [Vantage] ODBC Slow (Newbie Question)


The labordtl table is normally huge, at my last company it was hundreds of
thousands of records and anything I tried to do to it via odbc was slow.

Jeremy


Useful links for the Yahoo!Groups Vantage Board are: ( Note: You must have already linked your email address to a yahoo id to enable access. )
(1) To access the Files Section of our Yahoo!Group for Report Builder and Crystal Reports and other 'goodies', please goto: http://groups.yahoo.com/group/vantage/files/.
(2) To search through old msg's goto: http://groups.yahoo.com/group/vantage/messages
(3) To view links to Vendors that provide Vantage services goto: http://groups.yahoo.com/group/vantage/links



_____

Yahoo! Groups Links


* To visit your group on the web, go to:
http://groups.yahoo.com/group/vantage/


* To unsubscribe from this group, send an email to:
vantage-unsubscribe@yahoogroups.com <mailto:vantage-unsubscribe@yahoogroups.com?subject=Unsubscribe>


* Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service <http://docs.yahoo.com/info/terms/> .




[Non-text portions of this message have been removed]
If its access and you want me to take a look at it, while you have the query
design window open, switch to SQL View and send me the SQL command.
Send it to jeremyrleonard@....
I will be happy to fine tune it for you.
Thanks
Jeremy


[Non-text portions of this message have been removed]
I appreciate the offer and will be sending you off list. I am very curious and would like to learn as much as I can.

Juliet

-----Original Message-----
From: vantage@yahoogroups.com [mailto:vantage@yahoogroups.com]On Behalf
Of sarah.vareschi@...
Sent: Wednesday, May 18, 2005 8:30 AM
To: vantage@yahoogroups.com
Subject: RE: [Vantage] ODBC Slow (Newbie Question)


I can't imagine why this report is so slow on your systems if you have all
your joins correct. We have lots of ODBC reports on labor tables and they
range in speeds from instantaneous to maybe 30 seconds for reports with more
detail that span larger date ranges. Never has an odbc report taken several
minutes to run - and we do all of our reporting thru odbc and have for the
past year or so. If you want to try sending me your report
(sarah.vareschi@...) I will run it on our system to help you to
determine if the problem is with your report or something else (maybe odbc
config or network?).


Useful links for the Yahoo!Groups Vantage Board are: ( Note: You must have already linked your email address to a yahoo id to enable access. )
(1) To access the Files Section of our Yahoo!Group for Report Builder and Crystal Reports and other 'goodies', please goto: http://groups.yahoo.com/group/vantage/files/.
(2) To search through old msg's goto: http://groups.yahoo.com/group/vantage/messages
(3) To view links to Vendors that provide Vantage services goto: http://groups.yahoo.com/group/vantage/links
Yahoo! Groups Links
I am very intrigued with this. I would like to do this here. Our DB is over 7gb and am interested in any performance tweaks I can use. I will be looking this over in more detail.

Thanks.

JM

-----Original Message-----
From: vantage@yahoogroups.com [mailto:vantage@yahoogroups.com]On Behalf
Of Todd Caughey
Sent: Wednesday, May 18, 2005 8:30 AM
To: vantage@yahoogroups.com
Subject: RE: [Vantage] ODBC Slow (Newbie Question)


Even with all the best techniques for linking and filtering I have labor reports that take far too long (in RB but same would apply to Crystal). One other thing I often do is to pick an index for the table and make sure the filtering follows everything in the index PLUS set a filter for some given transaction number (in the index - primary index is best) that represents the start of my "range"....if the report is just for data this year then once a year I reset the start pointer to the first transaction of the year. When a filter follows an index like this is goes straight to that record without reading and testing all the prior records and then processes from there. On a huge table with multiple years this can cut the time dramaticaly. Takes a little annual tweaking but the difference in performance in January is HUGE compared to December so I know it helps even for one year.
-Todd C.

PS - I am currently developing benchmarks on various scenarios of links and filters for a presentation at the now "in-planning" User-for-User meeting Calvin Dekker is setting up for VMUG in Chicago in August. In addtion to my own "tricks" I am browsing the past group posts and pulling in any other info I can on optimizing reporting links and DB navigation.
It's crystal but I appreciate the offer very much. The people on this list are so wonderful.

-----Original Message-----
From: vantage@yahoogroups.com [mailto:vantage@yahoogroups.com]On Behalf
Of jeremyrleonard@...
Sent: Wednesday, May 18, 2005 8:32 AM
To: vantage@yahoogroups.com
Subject: Re: [Vantage] ODBC Slow (Newbie Question)


If its access and you want me to take a look at it, while you have the query
design window open, switch to SQL View and send me the SQL command.
Send it to jeremyrleonard@....
I will be happy to fine tune it for you.
Thanks
Jeremy
Thats fine, feel free to send me the rpt file.


[Non-text portions of this message have been removed]