View Issue Details

IDProjectCategoryView StatusLast Update
513RackTablesdefaultpublic2012-02-11 09:15
Reporteruser277Assigned To 
PrioritynormalSeveritytweakReproducibilityalways
Status closedResolutionno change required 
Product Version0.19.10 
Summary513: issue with id integer(10) auto_increment field in RackObject table
Descriptionid field in the RackObject table is unnecessary and causes downstream coding issues in heavily volatile environments.

Here is one example -
Our data center is a heavily virtualized volatile environment. We allow developers, testers and automated processes to build virtual machines, run UAT, build, security and stress scans on virtual machines and applications. Since we run multiple VMware large clusters we need to know where a virtual machine is in the cloud to deal with support issues that are raise by our users. RackTables is useful establish that tracking. The problem is we are building up the seq column really fast. Once the limit is hit we will have to normalize the table or change field type.

Another example -
Virtual machines in our data center are automatically registered into OMD/Check_mk/Nagios. We cannot create a notes_url directive using the following
http://racktables/index.php?page=object&tab=edit&object_name=ral_lin_vm_<dev_initials_such as abc>_123456

We've only been using RackTables for a few days and have built that field up surprisingly fast.


We are thinking of modifying the MySQL schema and the php sql to remove the dependency on the id field om all tables.

Do you guys have any other suggestions on how to deal with the situation?

Thanks,
/C
TagsNo tags attached.

Activities

user277

2012-01-27 21:28

  ~0000530

I mean to say...

/Chris

:)
infrastation

infrastation

2012-01-28 18:14

administrator   ~0000534

MySQL "unsigned int" type stands for 32 bits, that is, the maximum value is 4294967295. With even that being not enough, changing the type to "unsigned bigint" will ultimately resolve the problem of IDs rolling over.

Regarding the notes_url, could you clarify the scenario?
infrastation

infrastation

2012-02-10 13:15

administrator   ~0000593

This issue is likely to be closed due to lack of feedback.

user277

2012-02-10 13:32

  ~0000595

feedingback :)

Sorry I lost track of this bug. Our organization has been going through disaster recovery for the last 2 weeks.

Thanks,
/Chris C

user277

2012-02-10 14:01

  ~0000596

I dislike the idea of changing the column size. That is an easy way out will use more storage. It pushes the problem out into the future where the next admin will have to deal with it like when 80's programmers chopped of the first 2 digits of the year to save on storage space creating the Y2K problem.

I'm not the only shop that is heavily volatile. My college at a supercomputing center is interested in this product as well but their virtual environment moves 100% faster than ours. They rebuild 10,000+ node ROCKS cluster every few months and it would'nt take them long to hit the limitation either. They're physical infrastructure is intimidating also.


Nagios -
We use OMD (Open Monitor Distribution). OMD wrappers check_mk gui/configuration management, Nagios, pnp4nagios, nagvis, mysql, httpd, gearman into an easily managable package. OMD has made dealing with Nagios digestable and not "puke in your mouth"

I'd like to link our Nagios hosts to a cmdb and at first glance racktables looks like it might work but the frontend links to physical objects by sequence number and not hostname. ie....
http://racktables.example.org/index.php?page=object&object_id=189

Object_id 189 is a virtual machine wolft01 and it's scheduled to be broken down Feb 15, 2012. The next object id will be 5021. The url to this object will become...
http://racktables.example.org/index.php?page=object&object_id=5021

Any documentation linking to the object like wiki and Nagios will be out of date.

The php link should be
"http://racktables.akc.org/index.php?page=object&host=wolft01"

My OMD/Check_mk/Nagios configuration can then be...
extra_host_conf ["notes_url"] = [
  ( "http://racktables.example.org/index.php?page=object&host=$HOSTNAME$", ["linux"], ALL_HOSTS )
]

We have a one line configuration for so far 5000 hosts in our Nagios instance.

NOT.....
extra_host_conf ["notes_url"] = [
  ( "http://racktables.example.org/index.php?page=object&object_id=1", ["wolft01"] )
  ( "http://racktables.example.org/index.php?page=object&object_id=2", ["wolft02"] )
....
....
....
  ( "http://racktables.example.org/index.php?page=object&object_id=2", ["wolft99"] )

]

See how using object_id causes downstream static configurations?


The gist is the id column is not an effective primary key for the RackObjects table. If you use the name field for the primary key we have a nice tight table, no wasted space, no swiss cheese in the data as objects are removed and recreated, no limitations, and external linking is predictable.

Thanks,
/Chris C

ps...I'm not sure what happened with my Nagios configuration in the original post. It must've been parsed by Mantis into something wierd. Sorry about that.
adoom42

adoom42

2012-02-10 21:00

administrator   ~0000598

With an unsigned int column, if you added 10,000 objects per day, you would exhaust the available space in 1176 years. If you added 100,000 objects per day, it would take 117 years. Are you sure you're even close to reaching the int limit?

As for using the object name instead of the id in URLs, that could be done since the name is unique. NULL names are allowed so if you wanted to reference one of those by name using a URL there would be an issue.

But auto-incrementing numbers should still be used to identify rows on the database level. If you used only the device name and dropped the id column, all related tables would need to reference the name. That would consume much more space - vastly more than switching the column from int to bigint.

user277

2012-02-10 22:24

  ~0000599

We're not coming close to the limit yet but the rise in value is enough to be concerned. We're dealing with 1 moderately virtualized site at the moment. We haven't even got hardly moving at this site yet plus the plan is to bring up 3 more sites in the next 3 years. Calgon...

All the primary key unique key linking in mysql schema can be left up to you guys. I'm just saying its not a good idea in very large environments.


Really all that matters to me at the moment is that Nagios can call on a URL and get at the same object every time without having thousands of lines of static configuration or a perl script running from cron that queries mysql and generates Nagios config or rewriting code myself.

You could allow functions.php to query by name so then the url would become
 http://racktables.example.org/index.php?page=object&object_name=wolf12345
and then we don't care what the id number is or if it changes
infrastation

infrastation

2012-02-10 22:28

administrator   ~0000600

This script is likely to implement the requested functionality:

http://racktables.svn.sourceforge.net/viewvc/racktables/contribs/oneshot.php?revision=3904&view=markup
adoom42

adoom42

2012-02-10 22:49

administrator   ~0000601

The desired functionality is provided by the contrib mentioned. As for dropping the object id column and using the user-provided name column as a key, that will not be done. It is wrong on many levels and would require a tremendous amount of work to address a concern that realistically will never materialize (exceeding the int/bigint limit). Even if it did materialize, there are more feasible approaches such as using an auto-generated hexidecimal string instead of a number.
infrastation

infrastation

2012-02-11 09:15

administrator   ~0000608

There is an updated revision: http://racktables.svn.sourceforge.net/viewvc/racktables/contribs/oneshot.php?revision=4957&view=markup

Issue History

Date Modified Username Field Change
2012-01-27 21:26 user277 New Issue
2012-01-27 21:28 user277 Note Added: 0000530
2012-01-28 18:14 infrastation Note Added: 0000534
2012-01-28 20:44 infrastation Status new => feedback
2012-02-10 13:15 infrastation Note Added: 0000593
2012-02-10 13:32 user277 Note Added: 0000595
2012-02-10 14:01 user277 Note Added: 0000596
2012-02-10 21:00 adoom42 Note Added: 0000598
2012-02-10 22:24 user277 Note Added: 0000599
2012-02-10 22:28 infrastation Note Added: 0000600
2012-02-10 22:49 adoom42 Note Added: 0000601
2012-02-10 22:49 adoom42 Status feedback => closed
2012-02-10 22:49 adoom42 Resolution open => no change required
2012-02-11 09:15 infrastation Note Added: 0000608