Today's Messages (OFF)  | Unanswered Messages (ON)

Forum: MySQL
 Topic: mysql lost connection error
mysql lost connection error [message #3754] Tue, 02 December 2008 08:48
akshay_07  is currently offline akshay_07
Messages: 1
Registered: December 2008
Junior Member
i m running a delete query.it takes huge amount of time. after that it shows lost connection error.
after that mysql shows socket error.

plese help me.



 Topic: Migrate from oracle to mysql
Migrate from oracle to mysql [message #3753] Tue, 02 December 2008 05:35
andrea  is currently offline andrea
Messages: 1
Registered: December 2008
Junior Member
Hi All,

Pls, could you help me on the following issues:

I would like to migrate from oracle to mysql and one of important step is to
replace the actual job built on oracle environment;
Basically every day I receive from other oracle environment some 'oracle'
dump files (mainly CTL or oracle table exports). Today my oracle jobs load
the received data (CTL...) in my oracle tables. Now I would like to replace
my 'oracle' tables in MYSQL tables continuing to receive the file coming
from oracle environment.

So. Do you have same tools or artifacts to read the oracle CTL files (or
oracle tables dump) from mysql environment?
I already used mysqlimport GUI but I does not meet my needs. I need the
script/command to do these.

Thx in advance for prompt help/suggestion.
 Topic: iostat shows large amount of activity
iostat shows large amount of activity [message #3752] Sun, 30 November 2008 22:29
dcampano  is currently offline dcampano
Messages: 4
Registered: May 2008
Junior Member
Quick question. I have an LVM volume group set up with 2 logigal volumes. One of the volumes only contains the binary logs. When I watch the speed at which the logs grow with `ls -al` command, they are growing about 1MB every 3-4 seconds, yet iostat shows my write MB/s around 4-5. Anyone know why this could be?
 Topic: Slow-queries - Problems with mysql configuration
icon8.gif  Slow-queries - Problems with mysql configuration [message #3751] Sun, 30 November 2008 12:52
Breaks  is currently offline Breaks
Messages: 1
Registered: November 2008
Junior Member
Hello
I have portal on server:
Intel Bi Xeon Quad
8x 2.66+ GHz
12 MB L2 - FSB 1333 MHz
64 bits
RAM: 12 GB FBDIMM DDR2
Hard drives: 2x 147 GB SAS 15 000 tr/min - RAID 1 HARD

But sometimes i have problem with mysql. When it is working sometimes there are errors in slow-queris. I see there are lags for about 30 seconds and i can't find what is wrong. Look at STATUS in phpmyadmin.

http://hosting04.imagecross.com/image-hosting-11/2502info.pn g
I think this problem can be connected with these errors and my question is how can i fix them and where can be problem with slow-queries.

Here is my.cnf
http://phpfi.com/383720

Thanks for reply

P.S
sry for my english. I am begginer Wink

[Updated on: Sun, 30 November 2008 12:55]

 Topic: concurrent_insert=2 maintenace
concurrent_insert=2 maintenace [message #3750] Sat, 29 November 2008 16:51
blargman  is currently offline blargman
Messages: 1
Registered: November 2008
Junior Member
If I'm running concurrent_insert=2 and i end up with a bunch of deleted empty rows

what's the best way to clear those out?

shut down the server and run a myisamchk --sort ?

from what i've read concurrent_insert=2 , appends to the end of the table, ignoring deleted rows for storage. which would lead to empty deleted rows, aka a bunch of empty data in the .MYD . that's what i got from reading the docs on it anyway.



 Topic: Problem with INNODB -> BETWEEN AND + ORDER BY query
Problem with INNODB -> BETWEEN AND + ORDER BY query [message #3749] Fri, 28 November 2008 05:47
Juergen  is currently offline Juergen
Messages: 1
Registered: November 2008
Junior Member
Hi there,

I have a performance problem with inno.
I switched one table from MyISAM to INNODB because of transaction support:
the following query
SELECT * FROM table
WHERE artnr BETWEEN '120000' AND '130000'
AND menge IS NOT NULL
ORDER BY art ASC, p DESC


takes 0.8s to load with myisam, which is ok.
But with inno it takes 120+ sec (with heavy I/O activity).

The table contains only 500000 records. In both cases, no key is used.
But even if I add a key to artnr, the results are the same.
In my eyes this should not happen, even with no keys.

Another thing I don't understand:
MyISAM uses the index, INNODB not:
+----+-------------+-------+------+---------------+------+---------+------+--------+-----------------------------+
| id | select_type | table | type | possible_keys | key  | key_len | ref  | rows   | Extra                       |
+----+-------------+-------+------+---------------+------+---------+------+--------+-----------------------------+
|  1 | SIMPLE      | table | ALL  | TEST          | NULL | NULL    | NULL | 533968 | Using where; Using filesort |
+----+-------------+-------+------+---------------+------+---------+------+--------+-----------------------------+


If I change '130000' to something between '120000' and '128982' (or add a force index) the key is used with INNO but the performance is still very poor.

+----+-------------+-------+-------+---------------+------+---------+------+-------+-----------------------------+
| id | select_type | table | type  | possible_keys | key  | key_len | ref  | rows  | Extra                       |
+----+-------------+-------+-------+---------------+------+---------+------+-------+-----------------------------+
|  1 | SIMPLE      | fpos  | range | TEST          | TEST | 48      | NULL | 86134 | Using where; Using filesort |
+----+-------------+-------+-------+---------------+------+---------+------+-------+-----------------------------+


Does anybody know, why INNODB is so slow with this query?
I mean, maybe I can add a lookup myisam table or add an efficient index, but I don't understand what is happening in the background. I even could export all data manually to a csv file, sort the result with excel and reimport it to the database in less time (The "SELECT * FROM TABLE" statement takes 2 second to dump all data).

Oh, I forgot: The table:
CREATE TABLE `tableX` (
  `id` int(10) unsigned NOT NULL auto_increment,
  `art` int(11) unsigned NOT NULL default '0',
  `p` int(11) unsigned NOT NULL default '0',
  `artnr` varchar(6) character set latin1 default NULL,
  `name` varchar(45) character set latin1 default NULL,
  ...,
  PRIMARY KEY  USING BTREE (`id`),
  KEY `TEST` USING BTREE (`artnr`)
) ENGINE=InnoDB/MyISAM

Server = 5.1
 Topic: MYSQL DISTINCT
MYSQL DISTINCT [message #3747] Thu, 27 November 2008 02:17
daredavil82  is currently offline daredavil82
Messages: 1
Registered: November 2008
Location: Malaysia
Junior Member
hello everyone...

i like to know what is a suitable mysql clause instead of DISTINCT clause? i mean which clause same meaning and can produce same result as DISTINCT clause? how about GROUP BY? can i use GROUP by instead of DISTINCT clause?

 Topic: Possible memory leak in 5.0.67 b7
Possible memory leak in 5.0.67 b7 [message #3744] Wed, 26 November 2008 17:32
avinson  is currently offline avinson
Messages: 1
Registered: November 2008
Location: United States
Junior Member
I run a MySQL cluster with a single master and multiple slaves. All systems are running MySQL enterprise 5.0.56. I decided to experiment with the latest percona release on one of the slaves to see if there was a significant difference in performance. What I'm seeing is that the mysqld process slowly grows in memory size until eventually the system starts swapping. I thought it was possible that the percona release used slightly more memory overall so I reduced innodb buffer pool from 24G to 20G but I am still seeing the same behavior. It looks like there may be a memory leak or there is a setting that I need that I don't have. Here are my relevant config options:

skip-locking
key_buffer_size = 256M
max_allowed_packet = 16M
table_cache = 256
sort_buffer_size = 8M
myisam_sort_buffer_size = 8M

innodb_buffer_pool_size = 20G
innodb_log_file_size = 256M
innodb_file_per_table = 1
innodb_thread_concurrency = 0
innodb_file_io_threads = 10
innodb_max_dirty_pages_pct = 70
innodb_flush_method = O_DIRECT
innodb_support_xa = 0

query_cache_type = 1
query_cache_size = 64M
long_query_time = 1
max_connections = 2048
thread_cache_size = 32

With 24G innodb buffer pool size on the 5.0.56 enterprise binary the mysqld process is stable at 27G resident size in memory. The 5.0.67 percona binary steadily grows until it overwhelms the system. Is there a setting I'm missing? Any help is appreciated.

-Aaron
 Topic: Avoided filesort, but more rows to exmaine, need advice!
Avoided filesort, but more rows to exmaine, need advice! [message #3743] Wed, 26 November 2008 16:54
matthew016  is currently offline matthew016
Messages: 4
Registered: November 2008
Location: Brussels
Junior Member
Hi,

I am pretty new in optimizing tables with index and may need some help.
This is my query:

EXPLAIN SELECT timestamp
FROM Meting_INT_COPY
WHERE blockid = '200811252000'
ORDER BY timestamp DESC
LIMIT 1

If I have an index(blockid),
EXPLAIN will return the following information:

type possible_keys key rows Extra
ref index_blockid index_blockid 2638 Using where; Using filesort

If I add an index(blockid,timestamp)
EXPLAIN will display the following:

type possible_keys key
rows Extra
ref index_blockid,index_blockid_timestamp index_blockid_timestamp 8248
Using where; Using index


The index(blockid,timestamp) avoid the filesort + returns the result from index ! (Using where; Using index)
But why for the index(blockid) 2638 rows are returned to examine and for a more
specific index(blockid,timestamp) 8248 rows are returned ?

Thank you very much for any answer !
 Topic: MySQL functions
MySQL functions [message #3723] Thu, 20 November 2008 05:06
aloxy  is currently offline aloxy
Messages: 1
Registered: November 2008
Location: Uganda
Junior Member
who can bail me out with a code hat produces a result to the following question?

Return a list of all flights along with their current un-discounted price

according to these tables;

CREATE TABLE `fly-four-less`.`flight_profiles` (
`FLP_ID` decimal(8,0) NOT NULL default '0',
`FLT_NUMB` decimal(4,0) default NULL,
`SEATS` int(11) default NULL,
`PRICE_CODE` varchar(50) default NULL,
`from` varchar(50) default NULL,
`to` varchar(50) default NULL,
`MON` varchar(1) default NULL,
`TUE` varchar(1) default NULL,
`NEXT_DAY` int(11) default NULL,
`WED` varchar(1) default NULL,
`THU` varchar(1) default NULL,
`FRI` varchar(1) default NULL,
`SAT` varchar(1) default NULL,
`SUN` varchar(1) default NULL,
`depart` datetime default NULL,
`arrive` datetime default NULL,
PRIMARY KEY (`FLP_ID`),
UNIQUE KEY `FLT_NUMB` (`FLT_NUMB`),
KEY `FK_flight_profiles_1` (`from`),
KEY `FK_flight_profiles_2` (`to`),
CONSTRAINT `flight_profiles_ibfk_1` FOREIGN KEY (`from`) REFERENCES `airports` (`IATA_CODE`),
CONSTRAINT `flight_profiles_ibfk_2` FOREIGN KEY (`to`) REFERENCES `airports` (`IATA_CODE`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;

CREATE TABLE `fly-four-less`.`price` (
`PC_ID` int(11) NOT NULL default '0',
`PC_LEVEL` int(11) default NULL,
`PRICE` int(11) default NULL,
`FROM_DATE` datetime default NULL,
`TO_DATE` datetime default NULL,
PRIMARY KEY (`PC_ID`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;

CREATE TABLE `fly-four-less`.`discount` (
`DIS_ID` int(11) NOT NULL default '0',
`DAYS_ADV_MIN` int(11) default NULL,
`DAYS_ADV_MAX` int(11) default NULL,
`PASSENGER_TYPE` int(11) default NULL,
`DISCOUNT` int(11) default NULL,
PRIMARY KEY (`DIS_ID`),
KEY `FK_discount_1` (`PASSENGER_TYPE`),
CONSTRAINT `discount_ibfk_1` FOREIGN KEY (`PASSENGER_TYPE`) REFERENCES `person_type` (`PT_ID`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;

i came up with the following lines of query, but its not the results, if help avails can i have a deeper explanation on how the query digs to the table(s)?

SELECT * FROM FLIGHT_PROFILES F
WHERE F.NEXT_DAY=0;

Aloys
 Topic: How to retrieve Images from Outlook contacts(PST file) ?
How to retrieve Images from Outlook contacts(PST file) ? [message #3720] Wed, 19 November 2008 01:42
simjith  is currently offline simjith
Messages: 3
Registered: November 2008
Location: Qatar
Junior Member
Hi all,

I am a newbie to PHP + MySQL.
Now I am on a project in which I have to import Outlook contacts(PST file) to MySQL database and must provide a web interface for the same similar to Outlook Contact interface. By now I had completed the project except the importing of profile images and attached images.

If anyone of you have any idea,regarding how to extract the profile images and attached images from Outlook contacts PST file to MySQL/access/excel or any thing else, please share your idea

Thanx in advance

Simjith
 Topic: MySQL 5.1 version is stable in production environment
MySQL 5.1 version is stable in production environment [message #3719] Tue, 18 November 2008 04:15
bigrio  is currently offline bigrio
Messages: 6
Registered: January 2008
Location: Beijing China
Junior Member

I am considering to use partitioning function provided in MySQL 5.1 version.

But 5.1 is development version according to my check on mysql site.

5.0 is current production version.

Can I use 5.1 in production environment without serious bugs , problems?

Thanks for your opinion.
 Topic: Memcached Replication and Namespaces using MySQL Infrastructure.
Memcached Replication and Namespaces using MySQL Infrastructure. [message #3717] Sat, 15 November 2008 13:07
golanzakai  is currently offline golanzakai
Messages: 1
Registered: November 2008
Location: Israel
Junior Member
I would like to share with you guys a configuration method for memcached replication using mysql engine.

I wrote a blog entry describing it, I wish to hear feedback about the concept and if someone is already implemented such configuration on production environment.

http://golanzakai.blogspot.com/2008/11/memcached-replication -and-namespaces.html

Thanks
Golan Zakai

[Updated on: Sat, 15 November 2008 13:22]

 Topic: Index of CHAR(32) vs BINARY(16) vs TEXT
Index of CHAR(32) vs BINARY(16) vs TEXT [message #3715] Thu, 13 November 2008 08:51
jedwood  is currently offline jedwood
Messages: 1
Registered: November 2008
Junior Member
Things I don't care about:

- storage space (rows in the thousands, not millions)
- security (this is not a password)

Things I do care about:

- speed

So, I have a variable length bit of text that can be as small as 3 characters but will never be longer than what the TEXT type allows. And I need to make a comparison against this field. My idea is to md5 it in PHP and store it as either CHAR(32) or BINARY(16) (if I use the raw_output = TRUE parameter). But I could also skip the hash step and just store it in TEXT, though that seems like it'd be a much poorer index than one of the first two options.

Any thoughts?

Thanks!
 Topic: Maximum writes with MySQL
Maximum writes with MySQL [message #3713] Wed, 12 November 2008 11:04
v0idnull  is currently offline v0idnull
Messages: 1
Registered: November 2008
Junior Member
I'm thinking of installing Piwik, an open source alternative to Google Analytics. It'll be used on popular websites, sites that could potentially generate millions of hits a day.

Piwik stores its information into a MySQL database. So I'm wondering, how stable would MySQL be doing millions upon millions of writes a day? Size of the database I'm not too concerned about. Everyday reports will be generated and the data flushed, but during the day the db will be hit hard with writes.

Anyone have architectural choices? Different db engine for mysql? Maybe MySQL isn't up to this task?
 Topic: How to store temporary results best?
How to store temporary results best? [message #3712] Tue, 11 November 2008 11:43
Nyze1984  is currently offline Nyze1984
Messages: 1
Registered: November 2008
Junior Member
Dear Community

I need your advice Smile In our company we have a database with lots of tables and each table contains more then 500.000 rows.

My Job is to create a backend interface where user can select data via a filter interface and then keeping the results in a new table for later use. So far so good but i run into the problem that handling this large data takes a lot of time.

For Example
CREATE TABLE `searcg_xxtemp` (
  `id` bigint(20) unsigned NOT NULL auto_increment,
  `name` varchar(30) default NULL,
  `pw` varchar(30) default NULL,
  `email` varchar(255) default NULL,
  `isanonym` tinyint(4) NOT NULL default '0',
  `lastlogindatum` datetime NOT NULL default '0000-00-00 00:00:00',
  PRIMARY KEY  (`id`),
  UNIQUE KEY `name` (`name`),
  KEY `user_idkennung` (`id`,`kennung`),
  FULLTEXT KEY `pw` (`pw`)
) ENGINE=MyISAM SELECT * FROM user WHERE {Filterdata goes here}



This takes a lot of time because in worst case the SELECT statement returns 500.000 rows. How can handle this the best and fastest way. I need a good solution how to store the temporary results for later use! Maybe you can give me a good hint Smile
 Topic: Performance: Indexing double join with subselect
Performance: Indexing double join with subselect [message #3711] Tue, 11 November 2008 10:59
Dr.Ro.Se  is currently offline Dr.Ro.Se
Messages: 1
Registered: November 2008
Junior Member
UPDATE: SELECT statement changed (the first one was not wrong but foolish...

I have the following table of objects identified by columns 'objID' and 'stackVer' with data columns data1, data2 ... (much more) in a table "Object".

Data processing must be made with the data stored in data1, data2..., using a certain set of parameters defined in table "Recipe".

For processing, different configurations may be used, defined in table "Config", each configuration holds the "recipeID" of the parameter set that is currently active.

Now I need all rows from the "Object" table that have value 'prepared' set in column "processingStatus" in table "ObjectStatus", and the appropriate recipeID from table "Config" with "configType" set to e.g. 'Auto'.

Output should be like this:

objID   stackVer   data1   data2   data3   recipeID
---------------------------------------------------
10001          1    66.4     2.3       3        101


I hope this is understandable Wink

The SELECT statement below works, but it [...] is quite slow (several seconds with 1.4 million objects, there will be several billions later...)

Any idea how I can speed up this query with appropriate indexing and/or a more sophisticated SELECT statement and/or better table organizing?

Many thanks for help!

Robert



Tables:

drop table if exists Object;
create table Object
(
  objID          bigint not null,
  stackVer       tinyint not null,
  data1          real,
  data2          real,
  data3          real
);

create index idx_Object_objID on Object (objID);
create index idx_Object_stackVer on Object (stackVer);
create index idx_Object_objID_stackVer on Object (objID, stackVer);


drop table if exists ObjectStatus;
create table ObjectStatus
(
  objID              bigint not null,
  stackVer           tinyint not null,
  configID           tinyint,
  processingStatus   varchar(20) /* may be 'prepared', 'processing', 'processed' */
);

create index idx_ObjectStatus_objID on ObjectStatus (objID);
create index idx_ObjectStatus_stackVer on ObjectStatus (stackVer);
create index idx_ObjectStatus_objID_stackVer on ObjectStatus (objID, stackVer);
create index idx_ObjectStatus_processingStatus on ObjectStatus (processingStatus);


drop table if exists Config;
create table Config
(
  configID                 tinyint not null,
  configType               varchar(20), /* may be 'Auto', 'Manual', 'User1', 'User2'... */
  recipeID                 tinyint,
  configDescription        varchar(256)
);

insert into Config values (101, "Auto", 101, "Default");
insert into Config values (102, "Manual", 101, "Default");


drop table if exists Recipe;
create table Recipe
(
  recipeID                 tinyint not null,
  recipePath               varchar(256),
  recipeDescription        varchar(256)
);

insert into Recipe values (101, "/path/to/recipes/Default", "Default");



My current SELECT statement to do this (updated):

select Object.objID, Object.stackVer, Object.IcB1, Object.IcB2, Object.IcB3, a.recipeID from Object left outer join 
  (select Config.recipeID, ObjectStatus.objID, ObjectStatus.stackVer, ObjectStatus.processingStatus from ObjectStatus left outer join 
    Config on Config.configID=ObjectStatus.configID where ObjectStatus.processingStatus='prepared') a 
      on a.objID=Object.objID and a.stackVer=Object.stackVer where a.recipeID limit 2;


EXPLAIN says (updated):

+----+-------------+--------------+------+----------------------------------------------------------------+-----------------------------------+---------+--------------------+---------+-------------+
| id | select_type | table        | type | possible_keys                                                  | key                               | key_len | ref                | rows    | Extra       |
+----+-------------+--------------+------+----------------------------------------------------------------+-----------------------------------+---------+--------------------+---------+-------------+
|  1 | PRIMARY     | <derived2>   | ALL  | NULL                                                           | NULL                              | NULL    | NULL               | 1070000 | Using where |
|  1 | PRIMARY     | Object       | ref  | idx_Object_objID,idx_Object_stackVer,idx_Object_objID_stackVer | idx_Object_objID_stackVer         | 9       | a.objID,a.stackVer |      11 |             |
|  2 | DERIVED     | ObjectStatus | ref  | idx_ObjectStatus_processingStatus                              | idx_ObjectStatus_processingStatus | 23      |                    | 1176300 | Using where |
|  2 | DERIVED     | Config       | ALL  | NULL                                                           | NULL                              | NULL    | NULL               |       2 |             |
+----+-------------+--------------+------+----------------------------------------------------------------+-----------------------------------+---------+--------------------+---------+-------------+



[Updated on: Tue, 11 November 2008 12:49]

 Topic: Self-join and sorting
Self-join and sorting [message #3709] Tue, 11 November 2008 07:10
neilf  is currently offline neilf
Messages: 2
Registered: October 2008
Junior Member
Hi all.

I'm wondering if anyone has a solution for a problem I'm facing. I'm giving a simplified example for clarity (I hope), but I think it illustrates the problem.

Let's say I have a table called "Things" as follows:

Things
------
ThingID BIGINT
Timestamp BIGINT 
Attribute VARCHAR(20)


And there are two indexes on this table as follows:

Indexes
-----
AttributeTimestampIndex (Attribute, Timestamp)
ThingIDIndex (ThingID)


My table is denormalised, so each ThingID can have multiple Attributes for each timestamp, all stored in different rows.

However, importantly, each ThingID cannot have multiple timestamps.

For example:

Thing ID | Timestamp | Attribute
1        | 1         | 'foo'
1        | 1         | 'bar'
2        | 1         | 'foo'
2        | 1         | 'baz'
3        | 2         | 'foo'


Let's say I want to find the timestamp of all Things that have Attributes of 'bar' and 'foo', I execute the following query:

SELECT t1.Timestamp FROM Things t1, Things t2 WHERE t1.AttributeA = 'foo' AND t2.ThingID = t1.ThingID AND t2.AttributeA = 'bar' ORDER BY Timestamp


The query optimizer (correctly) figures out that 'bar' is a more selective attribute, and evaluates t1 first, then joins to t2 on the ThingIDIndex index.

The problem is, even though I have Timestamp in my index (which it should hopefully be able to use for sorting), MySQL ends up doing a Filesort, because I am selecting t1.Timestamp rather than t2.Timestamp (even though they are always the same, since there can't be multiple Timestamps for a ThingID).

The problem is I don't know in advance whether t1 or t2 will have the more selective clause applied to it, so I don't know whether to select from t1 or t2 in advance.

If I remove the t1 alias from Timestamp when I select it, then MySQL complains about an ambiguous column name (of course).

I can add the following clause to my SQL to enforce the fact that t1 and t2 have the same Timestamp.

t2.Timestamp = t1.Timestamp


But the effect is the same: MySQL still complains about an ambiguous column, and I still have to choose one of t1 or t2 to select from, thus I still have a 50% chance of MySQL having to do an unnecessary Filesort.

I'm not very keen on having to do a Filesort for a couple of reasons: 1. it's unnecessary work, and, more importantly 2. I can't start streaming the results to the client early, since I have to wait for the entire query to complete before I can get any results.

Anybody got any ideas as to how to solve this?
 Topic: SHOW PROCESSLIST not showing all users?
SHOW PROCESSLIST not showing all users? [message #3707] Mon, 10 November 2008 14:34
amandabee  is currently offline amandabee
Messages: 1
Registered: November 2008
Location: New York, NY
Junior Member

I'm trying to troubleshoot our mysql server, which keeps maxing out its connections. However, I can't get a full processlist for all users.

I'm logged in as root, and elsewhere as a client that has "process_priv" granted, but when I run "show processlist" I only see the list for the current user.

Am I missing something? All my reads of the documentation http://dev.mysql.com/doc/refman/5.0/en/show-processlist.html suggest that I ought to be able to see every process.
 Topic: MySQL crashes system
MySQL crashes system [message #3706] Mon, 10 November 2008 14:22
mani
Messages: 1
Registered: November 2008
Junior Member
Hi,

Environment:

CentOS 3.5 (2.4.21)
MySQL 5.0.64 Enterprise Server (Commercial)
2 GB RAM, 1 CPU

System variables:

bulk_insert_buffer_size 8388608
key_buffer_size 268435456
myisam_data_pointer_size 6
myisam_max_sort_file_size 2146435072
myisam_sort_buffer_size 67108864

We have an application which pumps in excess of 500 records/sec to MYISAM database and after 10 hrs or so, system silently crashes.After doing a power cycle, it is found the database has crashed and tables are corrupted.

<snip>
mysql> select * from xyz;
ERROR 1194 (HY000): Table 'xyz' is marked as crashed and should be repaired
<snip>

Also, we saw high iowait time (90%) for 1 min in every 3 min cycle.

Has anyone seen this problem before?

Thanks,
Mani.
 Topic: Change from MyISAM to InnoDB?
Change from MyISAM to InnoDB? [message #3704] Fri, 07 November 2008 08:58
armakuni  is currently offline armakuni
Messages: 3
Registered: July 2008
Junior Member
Usually people are talking about big databases but we have a very small database (~40MB, ~20 tables) where all tables are using MyISAM engine. My beginner question is: does it make sense to change to InnoDB engine? Current ratio between writes and reads from mysqlreport (last 80 days):
__ Questions ___________________________________________________________
Total         891.67M   128.0/s
  DMS         685.86M    98.4/s  %Total:  76.92
  Com_        111.21M    16.0/s           12.47
  COM_QUIT     97.60M    14.0/s           10.95
  -Unknown      3.00M     0.4/s            0.34
Slow 10 s           1     0.0/s            0.00  %DMS:   0.00  Log: OFF
DMS           685.86M    98.4/s           76.92
  SELECT      373.76M    53.6/s           41.92         54.50
  REPLACE     202.12M    29.0/s           22.67         29.47
  UPDATE      107.40M    15.4/s           12.04         15.66
  DELETE        2.42M     0.3/s            0.27          0.35
  INSERT      169.72k     0.0/s            0.02          0.02
Com_          111.21M    16.0/s           12.47
  set_option  102.20M    14.7/s           11.46
  stmt_prepar   2.99M     0.4/s            0.34
  stmt_execut   2.99M     0.4/s            0.34


and table lock status:

__ Table Locks _________________________________________________________
Waited          9.49M     1.4/s  %Total:   0.79
Immediate       1.19G   170.1/s


We are only interested about performance (speed) and reliability is not so important. Because there are lot of read and writes InnoDB should in theory be good if I understand correctly differences between MyISAM and InnoDB. But does the engine really matter in such a small database?
 Topic: Optimizing Query
Optimizing Query [message #3703] Fri, 07 November 2008 08:18
USAF_Pride
Messages: 1
Registered: November 2008
Junior Member
I have a database with 9 tables of unique data and 1 table that links it all together. I know the speed issue is all the JOINS and I am way outside of my league in attempting to fix it. The one table that links everything together only stores the primary ID from the other 9 tables. Here is the sql
SELECT DISTINCT guid, name as unique_name, ip as unique_IP, Violation, vdate, vtime
FROM tbl_player_kick 
LEFT JOIN tbl_guid ON tbl_player_kick.guid_id = tbl_guid.guid_id 
LEFT JOIN tbl_kick ON tbl_player_kick.kick_id = tbl_kick.kick_id 
LEFT JOIN tbl_name ON tbl_player_kick.name_id = tbl_name.name_id 
LEFT JOIN tbl_ip ON tbl_player_kick.ip_id = tbl_ip.ip_id 
LEFT JOIN tbl_violation ON tbl_player_kick.viol_type = tbl_violation.violation_id
LEFT JOIN tbl_date ON tbl_player_kick.date_id = tbl_date.date_id
LEFT JOIN tbl_time ON tbl_player_kick.time_id = tbl_time.time_id
WHERE tbl_player_kick.banned = 1


and the explain that goes with it

id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE tbl_player_kick ref banned banned 2 const 262650 Using where; Using temporary
1 SIMPLE tbl_guid eq_ref PRIMARY PRIMARY 4 aaser2_bansdbase.tbl_player_kick.guid_id 1
1 SIMPLE tbl_kick eq_ref PRIMARY PRIMARY 8 aaser2_bansdbase.tbl_player_kick.kick_id 1 Using index
1 SIMPLE tbl_name eq_ref PRIMARY PRIMARY 4 aaser2_bansdbase.tbl_player_kick.name_id 1
1 SIMPLE tbl_ip eq_ref PRIMARY PRIMARY 4 aaser2_bansdbase.tbl_player_kick.ip_id 1
1 SIMPLE tbl_violation eq_ref PRIMARY PRIMARY 8 aaser2_bansdbase.tbl_player_kick.viol_type 1
1 SIMPLE tbl_date eq_ref PRIMARY PRIMARY 4 aaser2_bansdbase.tbl_player_kick.date_id 1
1 SIMPLE tbl_time eq_ref PRIMARY PRIMARY 8 aaser2_bansdbase.tbl_player_kick.time_id 1

Any help would be greatly appreciated.

[Updated on: Fri, 07 November 2008 09:28]

 Topic: Need Help for Mysql-Query optimization
Need Help for Mysql-Query optimization [message #3702] Fri, 07 November 2008 07:26
yashvant  is currently offline yashvant
Messages: 1
Registered: November 2008
Junior Member
I have run this query make a more time please any body optimize this query as blew

$sql="SELECT f.feeder_code, f.subdivision_code FROM `feeder` f, sentout s
where ( f.start_date <= '2008-07-01' and f.end_date >= '2008-07-01' and
(s.month = 7 and s.year = 8 )) and f.feeder_code = s.feeder_code
UNION
SELECT feeder_code, subdivision_code FROM ht WHERE ht.month = 7 and
ht.year = 8 and concat( feeder_code, subdivision_code ) NOT IN
(SELECT concat( f.feeder_code, f.subdivision_code )
FROM `feeder` f, sentout s where (f.start_date <= '2008-07-01' and f.end_date >='2008-07-01' and (s.month = 7 and s.year = 8)) and f.feeder_code = s.feeder_code)
UNION
SELECT feeder_code, subdivision_code FROM lt WHERE lt.month = 7
and lt.year = 8 and concat( feeder_code, subdivision_code ) NOT IN
( SELECT concat( f.feeder_code, f.subdivision_code ) FROM feeder f, sentout s
where (f.start_date<= '2008-07-01' and f.end_date >= '2008-07-01' and
(s.month = 7 and s.year = 8)) and f.feeder_code = s.feeder_code
) AND concat( feeder_code, subdivision_code ) NOT IN ( SELECT concat( feeder_Code, subdivision_code )
FROM ht WHERE ht.month = 7 and ht.year = 8)
ORDER BY feeder_code, subdivision_code";

solved this problem because this take a more time

[Updated on: Fri, 07 November 2008 07:31]

 Topic: Optimizing simple order by query
Optimizing simple order by query [message #3686] Mon, 03 November 2008 18:53
novice.program  is currently offline novice.program
Messages: 1
Registered: November 2008
Junior Member
Hello I am having problems in optimizing a simple order by clause.

the query is:
explain SELECT `Entity`.`title`, `Entity`.`description`, `Entity`.`user_id`, `Entity`.`thumb_url`, `Entity`.`entity_uid`, `Entity`.`num_fachaks`, `Entity`.`number_of_views` FROM `entities` AS `Entity` ORDER BY `Entity`.`created` DESC LIMIT 21 \G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: Entity
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 11
Extra: Using filesort

I have index on the created filed by really dont understand that why mysql does not use it and prefers file sort.. the created field is a timestamp default current_timestamp.

Please help ... i am new to mysql and badly stuck.

Thanks.
 Topic: Increasing performance by parallel processing or multiple mysqld
Increasing performance by parallel processing or multiple mysqld [message #3685] Sat, 01 November 2008 06:18
nook  is currently offline nook
Messages: 1
Registered: November 2008
Junior Member
I'm running a search engine for realestate and I am in the process of acquiring a new server.

My search engine process is very well suited for parallelization, therefore I want to know how I can tweak mysql to use this in an optimal way.

I've noticed that on my present server mysql only uses one processor, the other is not used at all....

I see a possible number of possibilities:

1) putting different databases on different hard drives so read/write can be made fully parallel.

2) for each database running a different mysqld on its own port.

3) configuring my linux server so that it consist of (say) 4 virtual linux servers. Then on each of these 4 servers I install mysql for a different database.

I haven't been able to find any references on this subject.
 Topic: Choosing a server
Choosing a server [message #3679] Wed, 29 October 2008 19:56
carrigan  is currently offline carrigan
Messages: 1
Registered: October 2008
Junior Member
Hi guys,

I'm choosing a new server for the upcoming dedicated mysql server.

Few questions:

1) SCSI 15k, SCSI 10k, Raptor 10k in RAID 10?
2) Server options:

a) AMD Opteron 870, 8x2.0GhZ, 1000Mhz CPU, 8x1MB CPU Cache, DDR400 ECC Memory
b) Intel Xeon 5345, 8x2.33GhZ, 1333Mhz CPU, 2x8MB CPU Cache, FB-DIMM 533/677 Memory


I know it varies from application to application but let's assume for a general high load mysql server.

Thanks!
 Topic: PRIMARY key not working
PRIMARY key not working [message #3677] Wed, 29 October 2008 12:29
jeffStrahl  is currently offline jeffStrahl
Messages: 1
Registered: October 2008
Junior Member
Hello,
I have a query that is joining multiple tables. I have it optimized rather well, except for one table. I can't for the life of me figure out why that table is not using the pk as the key. It is causing the query to filesort and use a temp table, changing the query time from 0.003s to 23s, obviously too long.

Attached is a file with the database and query details.

Any assistance is greatly appreciated, for I don't know what it could be.

 Topic: Limits and datatype for connection_id()
Limits and datatype for connection_id() [message #3650] Fri, 24 October 2008 09:25
Poul  is currently offline Poul
Messages: 1
Registered: October 2008
Junior Member
Hi,

we use connection_id() return value to fill some fields in a mysql-myisam database.

I have spent a lot of time researching about the returned value, but I have not found any interesting information, so I'm asking this blog which I follow from one year now.

My questions are:

- select connection_id() returns an integer value: what are the limits? How should a declare the field to store it?

- what will happen when the limits are reached? Yes, it is possible; we are managing millions of connections per day on a single database...

- is it there any other better parameter available in a query, to ensure that two concurrent users will have returned different values?

Many thanks in advance.
 Topic: One to many -- Select 3 subitems for every item.
One to many -- Select 3 subitems for every item. [message #3649] Thu, 23 October 2008 09:12
wesleyh  is currently offline wesleyh
Messages: 1
Registered: October 2008
Junior Member
Hello,

I have two tables with a one to many relationship:

collection: id, user_id, title, description, item_count
items: id, collection_id, title, description

Now, I want to display the collections on a result page. (paginated). But, for each collection, I want to show the first 3 or so items that are in it.

What is the best possible way to do this efficiently? If I just do a straight up join I get the collection information 3 times as well, which I do not need.

Or should I just do a second query after having queried for the collections first? (Then using the collection_ids to get the items)?

What I do now:

SELECT c.id, c.title, c.description, GROUP_CONCAT(i.id) AS item_ids FROM collections c INNER JOIN items i ON i.collection_id = c.id ORDER by c.created_at DESC LIMIT 10

Then i collect those item_ids in php and do a "WHERE id IN (..)" on all those ids.

But I need to know if there is a better solution.

Thank you for your help,
Wesley
 Topic: slave sql thread always at 2 seconds
slave sql thread always at 2 seconds [message #3648] Thu, 23 October 2008 03:47
mancdaz  is currently offline mancdaz
Messages: 5
Registered: March 2008
Junior Member
How come the slave sql thread always shows a time of not less than 2:

30884 system user NULL Connect 2 Has read all relay log; waiting for the slave I/O thread to update it NULL

?
Daz
 Topic: Avoiding Stored Procedure Metadata Requests
Avoiding Stored Procedure Metadata Requests [message #3645] Wed, 22 October 2008 06:09
shlomoid  is currently offline shlomoid
Messages: 10
Registered: September 2007
Junior Member
Hi all,

Do you how to prevent the mysql connector/net asking the server for the stored procedure metadata each first execution? I can't seem to find any place that explains how to do that properly.

I've tried providing as much information about the SP parameters as possible, but nothing works.
Here is an example:

When launching a stored procedure from .net, I see this in the general log:
22 Init DB db_name
22 Query SELECT * FROM INFORMATION_SCHEMA.ROUTINES WHERE OUTINE_SCHEMA='db_name' AND ROUTINE_NAME='mySP'
22 Query SHOW CREATE PROCEDURE `db_name`.`mySP`
22 Query call mySP (10)


It makes sense to get the stored procedure code if the DAL needs to know the data types of the parameters, but it does that even when I am strong-typing the variables as much as I can.
In this case:
MySqlParameter param1 = new MySqlParameter("?p_param", MySqlDbType.Int32);
param1.Value = 10;
param1.Direction = ParameterDirection.Input;
param1.IsNullable = false;
param1.MySqlDbType = MySqlDbType.Int32;

cmd.Parameters.Add(param1);
cmd.ExecuteNonQuery();


Is there a correct way to do this that avoids those queries?

Thanks in advance!
Shlomo
 Topic: Optimization: Only InnobDB, no MyISAM
Optimization: Only InnobDB, no MyISAM [message #3636] Sat, 18 October 2008 18:06
colnector  is currently offline colnector
Messages: 1
Registered: October 2008
Junior Member
As I've chosen to use only InnoDB for my tables, I wish to know which configuration values I may cut to save on memory.

Also, I understand that MyISAM is used anyway for temporary tables. It also seems like the default "MySQL" and "Information_Schema" databases use MyISAM. Can this be changed or am I stuck with MyISAM anyway?

Thanks


Colnect Collectors Community

http://img.colnect.com/images/colnect_logo_255.png
 Topic: Max_data_length is Zero
icon3.gif  Max_data_length is Zero [message #3633] Thu, 16 October 2008 03:03
cowboymathu  is currently offline cowboymathu
Messages: 7
Registered: October 2008
Junior Member
Hi All,

show table status\G
*************************** 1. row ***************************
           Name: name_table
         Engine: MyISAM
        Version: 10
     Row_format: Dynamic
           Rows: 259956366
 Avg_row_length: 128
    Data_length: 33460714548
Max_data_length: 0
   Index_length: 44122680320
      Data_free: 3640
 Auto_increment: NULL
    Create_time: 2008-10-06 14:59:57
    Update_time: 2008-10-16 12:19:30
     Check_time: 2008-10-06 15:32:38
      Collation: latin1_swedish_ci
       Checksum: NULL
 Create_options: partitioned
        Comment: 


I have almost 260 millions of records with 73 GB disk capacity. What does actually "Max_data_length is zero" mean? How can I increase that?

Thanks,

[Updated on: Thu, 16 October 2008 03:03]

 Topic: Loading large tables with Unique Keys
Loading large tables with Unique Keys [message #3632] Wed, 15 October 2008 16:59
johndz  is currently offline johndz
Messages: 1
Registered: September 2008
Location: Broomfield, Colorado
Junior Member
This subject resurrects the trick shown in this thread for loading or altering large tables.

Alter table performance with MyISAM

I am struggling with some random cases where the REPAIR TABLE process is showing "Repair with keycache" instead of "Repair by sorting". In my latest test, on try # 2 it is "Repair by Sorting" which I believe is desired. Note that I use the "QUICK" option of the REPAIR TABLE command.

What would cause this to go into "Repair with keycache"?

Thanks,

johndz
 Topic: nothing
nothing [message #3630] Tue, 14 October 2008 11:05
chocobn69  is currently offline chocobn69
Messages: 1
Registered: October 2008
Junior Member
No Message Body

[Updated on: Thu, 30 October 2008 06:34]

 Topic: Table data covered entirely by indexes
Table data covered entirely by indexes [message #3629] Tue, 14 October 2008 10:48
neilf  is currently offline neilf
Messages: 2
Registered: October 2008
Junior Member
Hi all.

I have a table that is entirely covered by indexes. That is, every query that I want to perform on the table can be satisfied entirely from indexes, without having to read the table data at all.

This may seem like a bizarre thing to want to do, but does anyone know if it's possible to store the data in MySQL in *just* the indexes, without having to store the table data at all, since the table data is never actually accessed?

If this isn't possible in MySQL, does anyone know of any alternatives? I know that certain column-based stores (for example, C-store) might allow this, by storing my indexes as a set of projections over the data.

Thanks.

Neil.
 Topic: mysql performance on IBM Power6 systems
mysql performance on IBM Power6 systems [message #3624] Mon, 13 October 2008 15:44
fgarat  is currently offline fgarat
Messages: 1
Registered: October 2008
Location: Montevideo - Uruguay
Junior Member
Hi Peter and everyone,
im starting to evaluate the hadware plataform for a mysql database and i start wandering about how mysql perform on IBM new Power 6 systems as they are becaming more cheap and have the fatest clock per core.. so..
anyone have some info on this?.

AIX and mysql? Linux and mysql? mysql and power6?.


Thanks all,
Facundo.-
pd: sorry my bad english Smile
 Topic: sysbench
sysbench [message #3616] Wed, 08 October 2008 12:38
noah  is currently offline noah
Messages: 1
Registered: October 2008
Junior Member
I'd like to ask you guys something about sysbench, particularly about the OLTP complex test.

It seems that the test exercises deadlocks by design. The way the transactions are built seems to favor the occurrence of deadlocks, please correct me if I'm wrong.

So far I have made tests with MySQL and PostgreSQL, since both have MVCC the deadlocks didn't arise. Is there a way to "disable" MVCC on MySQL? my idea is to evaluate the performance of MVCC versus locking system.

Thanks! Smile

-Noah
 Topic: Comparing two query results
Comparing two query results [message #3609] Fri, 03 October 2008 16:38
mousey182  is currently offline mousey182
Messages: 1
Registered: October 2008
Junior Member
Hey Guys!

My first time here so hello all and thanks in advance for any help you may be able to give me.

I have a table called links with the following structure:

| link_id | type | frequency |

The type can be either link or advert.

I also have a table called links to keywords with the following structure:

| s_link_id | d_key_id |

I have a query that finds the most popular keywords by finding the keywords associated with the most frequently viewed links of type link.
I'm now trying to find the links of type advert that have keywords associated with them that match most closely the keywords of the links.

Any ideas how this can be achieved?

Ive tried the Like expression, but if the keywords associated with each type of link dont match exactlty then it doesnt return any results.

Thanks guys, I hope this makes sense!

Matt
 Topic: Memory tables in a separate db
Memory tables in a separate db [message #3604] Thu, 02 October 2008 10:25
mylesmg  is currently offline mylesmg
Messages: 1
Registered: October 2008
Junior Member
I have a medium-sized database with about 100M records in about 300 tables. The largest tables have at more 9M records. To help speed up some of the queries, which need to display real-time results, I have the system creating MEMORY tables (TEMPORARY tables didn't work for me). I have about 7K MEMORY tables, and I was wondering if putting these temporary tables into a separate database schema would make a performance difference.

Currently I'm performing an average of 125 queries per second. The entire database fits into memory (INNODB buffer pool) and seems to perform quite well overall.

My concern is that when I have to start using replication I may encounter issues with these temporary tables. MySQL Cluster is not available as an option on Windows yet, and since that's all I know how to administer I have to keep using it.

So what do we know about the performance of tables being accessed across databases and the performance impact of this?

Thanks in advance,

Michael Myles
www.aimcrm.com
Pages (6): [1  2  3  4  5  6    »]



Current Time: Tue Dec 2 16:19:58 EST 2008

Total time taken to generate the page: 0.58402 seconds
.:: Contact :: Home :: MySQL Support by Percona.com ::.

Powered by: FUDforum 2.7.5.
Copyright ©2001-2006 FUD Forum Bulletin Board Software

MySQL Performance | Forum authority Badge