Oramoss Blog

Oracle Optimized Warehouse Initiative (OWI)

15 April 2008, 8:53 pm

I enjoyed a trip out of the office today with my manager. We went down to the Oracle Enterprise Technology Centre, in Reading, to hear about the Oracle Optimized Warehouse Initiative. It was basically a half day pitch from Oracle and Sun today, although there are other dates with different hardware vendors (IBM, HP and Dell).

It was an interesting day, although I didn’t really hear anything new, per se. I think the main things I took away from the session were:

The availability of a set of reference configurations providing matrices which cover various permutations of user count, storage cost focus, warehouse size and hardware vendor.
The possibility of using the reference configurations either minimally, to simply cross check a proposed hardware specification for a given workload, to ensure it seems appropriate, or going the whole hog and using an “out of the box” reference configuration, delivered to your office, fully configured with all software installed, in a box, ready to run in your data.
Oracle are pushing RAC and ASM heavily in the DW space – no surprise there.
HammerOra and ORION are used by Oracle and the hardware vendors to assess the reference configurations…and there is nothing stopping you using them for your own benchmarking efforts

It was interesting to hear about the Proof Of Concept facility that Sun has, in Linlithgow, Scotland. The facility allows Sun (and an Oracle customer) to take a server off the production line and, working with the customer, test their workload on that machine to see if it’s going to work or not. Neat, and since we’re going to be using some Sun kit for a DW shortly, it sounds like an opportunity.

Funniest thing of the day for me, was the last slide in the pitch given by Mike Leigh of Sun which had the title “The Power of 2″ and was illustrating the benefits to customers of Oracle and Sun as a united force. I didn’t really take much notice, as I was too busy smiling, as I looked at the title and it made me think of Doug and his Parallel Execution and the ‘Magic of 2′ paper (the Magic of 2 bit actually being from this paper by Cary).

If you’re building a warehouse, or just want to get an idea of whether your hardware is appropriate for the job, it’s probably worth reading up on the OWI.

Tagged ASM, DW, RAC

Comments Off

Problem with _gby_hash_aggregation_enabled parameter

10 April 2008, 9:58 pm

Here’s a tale about an Oracle initialisation parameter…and a lesson we should all take note of…

For three days, my colleagues in the support team on one of the warehouses I’m involved in, were struggling with a piece of code which was exhausting the available temp space and after trying everything they could think of, they asked me to take a look. I must admit I was a little baffled at first because the piece of code in question had been happily running for some time now and every time I’d run it, I’d never noticed that TEMP was anywhere near being exhausted so, whilst I could tell the process had some kind of problem, I was in the dark as to exactly what that problem was.

After trying to break down the large query into several smaller steps, I realised that it was an early step in the query that was exhibiting the problem of temp exhaustion – the step being a pivot of around 300 million measure rows into a pivot target table.

This process runs monthly and it had run without issue on all of the previous 20 occurrences and for roughly the same, if not more rows to pivot…so it didn’t appear to be the volume that was causing the problem.

There had been no changes in the code itself for many months and the current version of the code had been run successfully in previous months, so it didn’t appear to be a code fault.

I obtained the actual execution path for the statement whilst it was running and it looked reasonable, although looking at it, triggered a thought in my mind…what if something had changed in the database configuration?

Why would I get that thought from looking at the execution path?

Well, a while back, we had received some advice that a line in our init.ora as follows, should be changed to set the feature to TRUE instead of FALSE, so that the feature became active:

_gby_hash_aggregation_enabled = FALSE

This results in a line in the plan that reads:

HASH GROUP BY

instead of

SORT GROUP BY

The parameter was set to FALSE due to a known bug and the issues we’d seen with it, however the recent advice we’d received, indicated that the bug had been resolved at the version level we were on and that by enabling the feature – which enables GROUP BY and Aggregation using a hash scheme – we’d gain a performance boost for certain queries.

So, the DBA team researched the advice and it appeared to be the case, that the bug (4604970) which led to the disabling of the feature was fixed at our version level (10.2.0.3 on HP-UX). We duly turned on the feature in a pre production environment and ran it for a while without noticing any issues. We then enabled it in production and again, for a while, we’ve not noticed any issues…until now.

After a check back through the logs, it appeared that since the parameter was first enabled, the queries which were now failing, had not been run at all…they had only run prior to the parameter change…so with my suspicions aroused further, I disabled the feature at the session level and reran the process. It completed in a normal time frame and used a small amount of TEMP – hooray!

So, now we have to go back to support to try and understand if the original bug is not quite fixed or whether this is a different scenario…in any event, we’re going to disable the feature for now, even though we’re only getting problems with the feature on 2 processes out of perhaps thousands.

So, what’s the lesson to learn?

Well, quite simply, that you need to have a thorough audit of what configuration changes you’ve made together with a good audit of the processes you’ve run so that you can work out what has changed since the last time you successfully ran a process. This gives you a fighting chance of spotting things like the above.

Tagged bugs, dba, tuning

Comments Off

Tracking TEMP usage throughout the day

26 February 2008, 7:34 pm

I really wanted to do this via AWR, but I’ve not been able to find out if this kind of information is stored and, if it is, how I’d get access to it…maybe someone else knows…hint hint?!

I have various queries I run whilst I’m online and processes are running, but what I really wanted, was to know the usage profile throughout the day…so, given that I couldn’t find an AWR way of tracking our use of TEMP on a database, I figured I’d use a more klunky method…

DROP TABLE mgmt_t_temp_use_history PURGE/

CREATE TABLE temp_use_history(snap_date      DATE         NOT NULL                             ,sid            NUMBER       NOT NULL                             ,segtype        VARCHAR2(9)  NOT NULL                             ,qcsid          NUMBER       NULL                             ,username       VARCHAR2(30) NULL                             ,osuser         VARCHAR2(30) NULL                             ,contents       VARCHAR2(9)  NULL                             ,sqlhash        NUMBER       NULL                             ,sql_id         VARCHAR2(13) NULL                             ,blocks         NUMBER       NULL                             )PCTFREE 0COMPRESSNOLOGGING/CREATE UNIQUE INDEX tuh_pk ONtemp_use_history(snap_date,sid,segtype)PCTFREE 0COMPRESSNOLOGGING/ALTER TABLE temp_use_history ADD CONSTRAINT tuh_pk PRIMARYKEY(snap_date,sid,segtype)USING INDEX/

DECLARE  l_program_action VARCHAR2(2000);  l_27477 EXCEPTION;  PRAGMA EXCEPTION_INIT(l_27477,-27477); BEGIN  BEGIN    DBMS_SCHEDULER.CREATE_SCHEDULE(      schedule_name   => 'MINUTELY_5M'     ,start_date      => SYSDATE     ,repeat_interval => 'FREQ=MINUTELY;INTERVAL=5'     ,comments        =>'Daily schedule to run a job every five minutes.'                                  );  EXCEPTION    WHEN l_27477 THEN      NULL;  -- Ignore if the schedule exists  END;

  l_program_action :=                   'DECLARE';  l_program_action := l_program_action||'  l_date DATE := SYSDATE; ';  l_program_action := l_program_action||'BEGIN';  l_program_action := l_program_action||'  INSERT /*+ APPEND */ INTO temp_use_history(snap_date,sid,qcsid,username,osuser,contents,segtype,sqlhash,sql_id,blocks)';  l_program_action := l_program_action||'  SELECT l_date,s.sid,ps.qcsid,s.username,s.osuser,su.contents,su.segtype,su.sqlhash,su.sql_id,sum(su.blocks)';  l_program_action := l_program_action||'  FROM   v$sort_usage su';  l_program_action := l_program_action||'  ,      v$session s';  l_program_action := l_program_action||'  ,      v$px_session ps';  l_program_action := l_program_action||'  WHERE  s.sid=ps.sid(+)';  l_program_action := l_program_action||'  AND    s.saddr =su.session_addr';  l_program_action := l_program_action||'  AND    s.serial# =su.session_num';  l_program_action := l_program_action||'  GROUP BY s.sid,ps.qcsid,s.username,s.osuser,su.contents,su.segtype,su.sqladdr,su.sqlhash,su.sql_id;';  l_program_action := l_program_action||'  COMMIT; ';  l_program_action := l_program_action||'END;';

  BEGIN    DBMS_SCHEDULER.CREATE_PROGRAM(      program_name => 'SNAP_TEMP_USAGE'     ,program_type => 'PLSQL_BLOCK'     ,program_action => l_program_action     ,enabled        => TRUE     ,comments       => 'Program to snap the temp usage into TEMP_USE_HISTORY'                                 );  EXCEPTION    WHEN l_27477 THEN      NULL;  -- Ignore if the program exists  END;

  BEGIN    DBMS_SCHEDULER.CREATE_JOB(      job_name      => 'JOB_SNAP_TEMP_USAGE'     ,program_name  => 'SNAP_TEMP_USAGE'     ,schedule_name => 'MINUTELY_5M'     ,enabled       => TRUE     ,auto_drop     => FALSE     ,comments      => 'Job to snap the temp usage into TEMP_USE_HISTORY'                             );  EXCEPTION    WHEN l_27477 THEN      NULL;  -- Ignore if the job exists  END;END;/

I can now run queries against the TEMP_USE_HISTORY table to show how much TEMP has been used, when, by whom and for what use, e.g.

SQL> edWrote file afiedt.buf

  1  select snap_date,round(sum(blocks)*32/1024/1024) gb  2  from temp_use_history  3  where snap_date > sysdate-1  4  group by snap_date  5  having round(sum(blocks)*32/1024/1024) > 50  6* order by 1SQL> /

SNAP_DATE                    GB-------------------- ----------26-FEB-2008 16:12:25         5726-FEB-2008 16:17:25         6526-FEB-2008 16:22:25         7426-FEB-2008 16:27:25         8626-FEB-2008 16:32:25         9526-FEB-2008 16:37:25        10726-FEB-2008 16:42:25        12126-FEB-2008 16:47:25        12726-FEB-2008 16:52:25        14726-FEB-2008 16:57:25        16026-FEB-2008 17:02:25        16226-FEB-2008 17:07:25        17926-FEB-2008 17:12:25        19626-FEB-2008 17:17:25        20826-FEB-2008 17:22:25        21726-FEB-2008 17:27:25        23326-FEB-2008 17:32:25        24126-FEB-2008 17:37:25        25126-FEB-2008 17:42:25        25726-FEB-2008 17:47:25        26226-FEB-2008 17:52:25        26426-FEB-2008 17:57:25        26726-FEB-2008 18:02:25        20127-FEB-2008 00:27:25         5927-FEB-2008 00:32:25         6027-FEB-2008 00:37:25         6927-FEB-2008 01:12:25         5727-FEB-2008 02:22:25         5327-FEB-2008 09:57:25         51

29 rows selected.

From the above, I can see that, during the last twenty four hours on the database in question, there was reasonably heavy use of the TEMP area between 4pm and 6pm yesterday, and that the load peaked at approximately 267Gb.

Tagged awr, code, dba

Comments Off

Getting multi row text items from Designer repository

21 February 2008, 11:06 pm

I’ve mentioned Lucas Jellema before when talking about Oracle Designer – he’s helped me out again today via this post on the AMIS blog.

What I wanted, was to be able to extract, using a SQL query against the designer repository, the “Description” attribute from our Tables so that I could create table comments in the database which had the description from the table in Designer.

Extracting scalar attributes is simple enough, however, description is a multi row text property so it needed a bit more thought…and rather than reinvent the wheel, I had a look at Lucas’ stuff and sure enough found the post above. It was almost what I wanted, only I had to change it to look for table stuff rather than entities and attributes.

I used the same TYPE and sum_string_table function as Lucas so if you’re trying to use this stuff you’ll probably need to read Lucas’ article first.

The query I ended up with is below…it’s been “sanitized” somewhat, but I’m sure you’d get the picture, if retrieving stuff out of the designer repository is a requirement of yours.

WITH dsc AS(SELECT txt.txt_text,      txt.txt_refFROM   cdi_text txtWHERE  txt.txt_type = 'CDIDSC'), add_cast AS(SELECT appsys.name application_system_name,      b.name table_name,      b.alias,      CAST(COLLECT(dsc.txt_text) AS string_table) tab_descriptionFROM   dsc,      designer.ci_application_systems appsys,      designer.ci_app_sys_tables a,      designer.ci_table_definitions bWHERE  dsc.txt_ref = a.table_referenceAND    b.irid = a.table_referenceAND    a.parent_ivid = appsys.ividGROUP BY appsys.name ,        b.name,        b.alias)SELECT application_system_name,      table_name,      alias,      sum_string_table(tab_description)FROM   add_castWHERE  application_system_name = 'MY_APP_SYS'and    table_name = 'MY_TABLE_NAME'/

Thanks Lucas!

On another note, regular visitors may realise I’ve now got my own oramoss dot com domain and that my blogger blog is published on that domain now.

Thanks to Andreas Viklund for the template.

If anyone sees anything untoward with the new site please feel free to drop me a note. It’s a bit thin on content but I’ll work on that over time.

Tagged blog, code, designer

2 Comments

Start the new year with a game of tag!

9 January 2008, 7:09 pm

Thanks to Doug, I’ve been “tagged” – he’d obviously noticed I’d been sleeping, blog-wise, for some time and decided that I should come out of hibernation!

The idea behind the tagging being to “learn more about the people you interact with using New Web”.

So, here are eight pieces of trivia about myself:

1. I am the son of a couple of geordies. My dad being a miner and my mum being a machinist and taxi driver in her time. Mum and dad were married at Gretna Green. Mum is also fairly rare in that she is a successful heart transplant patient of nearly ten years.

2. I love animals – all animals really, but in particular dogs, and in particular Boxer dogs, and even more particularly white boxer dogs – and no, to scotch a common myth, not all white boxer dogs are deaf – it’s the same percentage as all breeds of dog that are deaf – around 20%.

3. I’m not a vegetarian, despite my love of animals, however my wife is pretty much a vegetarian…which obviously makes life interesting in the kitchen department.

4. If I wasn’t an Oracle person then I’d be a chef. In all truth, the only reason I haven’t switched from being an Oracle person to a chef is that I don’t think I could earn as much money in that field. Yes, Gordon Ramsey earns millions but it’s taken him decades to get there and I’m not sure I’m as capable in that field as I am in the Oracle one. So the plan will be to work in Oracle until such time as I think I’ve comfortable and then I’ll switch to something in the arena of food and drink.

5. I love car racing – particularly Formula 1. Some wifes are “Golf Widows” – mine is an “F1 Widow” and calls me a “saddo” whenever I’m surfin’ on my laptop in the evening and she spots me looking at the ITV 1 Formula 1 website. Obviously I think I could go a fast as Lewis Hamilton but I’m all too aware that I’d probably end up looking like Richard Hammond if I tried!

6. I’m addicted to reading – both fiction and non. My ideal holiday being one where I just sit by the pool/on the beach, reading constantly for a fortnight.

7. Speaking of ideal holiday…my favourite place is the carribean, but then what’s not to like there! My favourite island being St Lucia since that’s where I got married to Amanda.

8. I have a burning desire to live in another country – preferably somewhere hot. Whether my desire is fulfilled who knows…watch this space!

That’s my eight things…so over to a new set of people to continue the chain…I’ve no idea if these folk have been tagged or not yet – apologies if they have and the chain isn’t extended but ultimately, isn’t that inevitable?

Kevin Closson
Tanel Poder
Nicholas Goodman
Mogens Nogaard
Anjo Kolk
Pete Finnigan – how ironic with this tagging being effectively a virus!
Eric S Emrick
Christian Bilien

Tagged social

Comments Off

CALIBRATE_IO Poll

25 September 2007, 10:17 pm

After Doug’s comment on my earlier blog on CALIBRATE_IO, and after seeing Kevin using Mr Poll to do a RAC poll, I’ve now created a poll for the results people are getting from this CALIBRATE_IO routine in 11g.

If you’d like to contribute please visit using the following link:

Oracle 11g CALIBRATE_IO Results

No prizes for the “biggest”…Kevin is bound to have something in his lab that beats us all!

Thanks

Tagged dba, poll, tuning

2 Comments

11g IO Calibration tool

25 September 2007, 11:58 am

After reading how inadequate Doug was feeling over his IO subsystem, I decided to see how quick mine was…not that we’re getting into a “mine is better than yours” game, but rather to see how mine stacks up against Doug’s, bearing in mind his is a 5 disk stripe natively attached to his machine (I’m assuming) and mine is a logical disk attached to a VMWare machine…although admittedly, the PC underneath this logical disk is running, motherboard based, RAID striping, over two physical SATA disks…I just figured it would be interesting to compare.

Obviously, any experiment that goes flawlessly according to a preconceived plan is:

1. Boring
2. Less educational
3. Not normally one I’ve done – mine always have problems it seems!

I ran the calibration on my VMWare based OpenSuse 10 linux with Oracle 11g and it immediately came up with a problem:

SQL> @ioSQL> SET SERVEROUTPUT ONSQL> DECLARE2    lat  INTEGER;3    iops INTEGER;4    mbps INTEGER;5  BEGIN6  -- DBMS_RESOURCE_MANAGER.CALIBRATE_IO (, , iops, mbps, lat);7     DBMS_RESOURCE_MANAGER.CALIBRATE_IO (2, 10, iops, mbps, lat);89     DBMS_OUTPUT.PUT_LINE ('max_iops = ' || iops);10     DBMS_OUTPUT.PUT_LINE ('latency  = ' || lat);11     dbms_output.put_line('max_mbps = ' || mbps);12  end;13  /DECLARE*ERROR at line 1:ORA-56708: Could not find any datafiles with asynchronous i/o capabilityORA-06512: at "SYS.DBMS_RMIN", line 453ORA-06512: at "SYS.DBMS_RESOURCE_MANAGER", line 1153ORA-06512: at line 7

Of course, consulting the manual led me to run this query:

SELECT name, asynch_ioFROM v$datafile f,v$iostat_file iWHERE f.file#        = i.file_noAND   filetype_name  = 'Data File'/

which gave:

SQL> /

NAME                                               ASYNCH_IO-------------------------------------------------- ---------/home/oracle/oradata/su11/system01.dbf             ASYNC_OFF/home/oracle/oradata/su11/sysaux01.dbf             ASYNC_OFF/home/oracle/oradata/su11/undotbs01.dbf            ASYNC_OFF/home/oracle/oradata/su11/users01.dbf              ASYNC_OFF/home/oracle/oradata/su11/example01.dbf            ASYNC_OFF

…or in other words no asynchronous IO available – as the error message had said.

After altering the filesystemio_options parameter to “set_all” and bouncing the instance, a second run of the calibration process seemed to work fine…

SQL> @ioSQL> SET ECHO ONSQL> SET SERVEROUTPUT ONSQL> DECLARE2    lat  INTEGER;3    iops INTEGER;4    mbps INTEGER;5  BEGIN6  -- DBMS_RESOURCE_MANAGER.CALIBRATE_IO (, , iops, mbps, lat);7     DBMS_RESOURCE_MANAGER.CALIBRATE_IO (2, 10, iops, mbps, lat);89     DBMS_OUTPUT.PUT_LINE ('max_iops = ' || iops);10     DBMS_OUTPUT.PUT_LINE ('latency  = ' || lat);11     dbms_output.put_line('max_mbps = ' || mbps);12  end;13  /max_iops = 72latency  = 13max_mbps = 26

PL/SQL procedure successfully completed.

So,my figures are considerably lower than those Doug achieved:

max_iops = 112latency  = 8max_mbps = 32

…but not too bad I guess considering the fact that mine is a VM and the hardware I’m running is more humble…no seriously, size does not matter!

Tagged dba, errors, tuning

4 Comments

11g PX tracefiles now have the tracefile identifier on them

4 September 2007, 9:37 pm

Now that I’ve got 11g up and running on OpenSuse 10.2 on a VMWare 6 VM, I’ve had time to do some playing with the latest and greatest release and the first thing I’ve noticed, when running some of Doug’s PX test scripts, is that the trace files generated for PX slaves now have the Tracefile Identifier appended to their name, making it easier to see which OS Process (PID) was responsible for the creation of the trace file – makes things a little easier and clearer.

In 10gR2 (10.2.0.2.0 specifically) the trace files would come out with names in this format:

_ _.trc

e.g. fred_p001_6789.trc

In 11gR1 (11.1.0.6.0 specifically) the trace files come out with names in this format:

_ __
.trc

e.g. fred_p001_5678_jeff.trc

This assumes you’ve set the tracefile identifier in the first place, otherwise that bit won’t be present. Use the following to set it, choosing whatever identifier you require of course:

alter session set tracefile_identifier='jeff';

It was interesting that the location of such files has also changed due to the implementation of Automatic Diagnostic Repository (ADR). More information on that here.

Tagged dba, tuning

4 Comments

Oramoss Blog

Oracle Optimized Warehouse Initiative (OWI)

Problem with _gby_hash_aggregation_enabled parameter

Tracking TEMP usage throughout the day

Getting multi row text items from Designer repository

CALIBRATE_IO Poll

11g IO Calibration tool

11g PX tracefiles now have the tracefile identifier on them

Recent Posts

Recent Comments

May 2012
M	T	W	T	F	S	S
« Oct
	1	2	3	4	5	6
7	8	9	10	11	12	13
14	15	16	17	18	19	20
21	22	23	24	25	26	27
28	29	30	31

Recent Posts

Recent Comments

Tags