Today's Messages (OFF)
| Unanswered Messages (ON)
| Forum: MySQL |
|---|
| Topic: mysql lost connection error |
|---|
| mysql lost connection error [message #3754] |
Tue, 02 December 2008 08:48 |
akshay_07 Messages: 1 Registered: December 2008 |
Junior Member |
|
|
i m running a delete query.it takes huge amount of time. after that it shows lost connection error.
after that mysql shows socket error.
plese help me.
|
|
| | Topic: Migrate from oracle to mysql |
|---|
| Migrate from oracle to mysql [message #3753] |
Tue, 02 December 2008 05:35 |
andrea Messages: 1 Registered: December 2008 |
Junior Member |
|
|
Hi All,
Pls, could you help me on the following issues:
I would like to migrate from oracle to mysql and one of important step is to
replace the actual job built on oracle environment;
Basically every day I receive from other oracle environment some 'oracle'
dump files (mainly CTL or oracle table exports). Today my oracle jobs load
the received data (CTL...) in my oracle tables. Now I would like to replace
my 'oracle' tables in MYSQL tables continuing to receive the file coming
from oracle environment.
So. Do you have same tools or artifacts to read the oracle CTL files (or
oracle tables dump) from mysql environment?
I already used mysqlimport GUI but I does not meet my needs. I need the
script/command to do these.
Thx in advance for prompt help/suggestion.
|
|
| | Topic: iostat shows large amount of activity |
|---|
| iostat shows large amount of activity [message #3752] |
Sun, 30 November 2008 22:29 |
dcampano Messages: 4 Registered: May 2008 |
Junior Member |
|
|
|
Quick question. I have an LVM volume group set up with 2 logigal volumes. One of the volumes only contains the binary logs. When I watch the speed at which the logs grow with `ls -al` command, they are growing about 1MB every 3-4 seconds, yet iostat shows my write MB/s around 4-5. Anyone know why this could be?
|
|
| | Topic: Slow-queries - Problems with mysql configuration |
|---|
Slow-queries - Problems with mysql configuration [message #3751] |
Sun, 30 November 2008 12:52 |
Breaks Messages: 1 Registered: November 2008 |
Junior Member |
|
|
Hello
I have portal on server:
Intel Bi Xeon Quad
8x 2.66+ GHz
12 MB L2 - FSB 1333 MHz
64 bits
RAM: 12 GB FBDIMM DDR2
Hard drives: 2x 147 GB SAS 15 000 tr/min - RAID 1 HARD
But sometimes i have problem with mysql. When it is working sometimes there are errors in slow-queris. I see there are lags for about 30 seconds and i can't find what is wrong. Look at STATUS in phpmyadmin.
http://hosting04.imagecross.com/image-hosting-11/2502info.pn g
I think this problem can be connected with these errors and my question is how can i fix them and where can be problem with slow-queries.
Here is my.cnf
http://phpfi.com/383720
Thanks for reply
P.S
sry for my english. I am begginer
[Updated on: Sun, 30 November 2008 12:55]
|
|
| | Topic: concurrent_insert=2 maintenace |
|---|
| concurrent_insert=2 maintenace [message #3750] |
Sat, 29 November 2008 16:51 |
blargman Messages: 1 Registered: November 2008 |
Junior Member |
|
|
If I'm running concurrent_insert=2 and i end up with a bunch of deleted empty rows
what's the best way to clear those out?
shut down the server and run a myisamchk --sort ?
from what i've read concurrent_insert=2 , appends to the end of the table, ignoring deleted rows for storage. which would lead to empty deleted rows, aka a bunch of empty data in the .MYD . that's what i got from reading the docs on it anyway.
|
|
| | Topic: Problem with INNODB -> BETWEEN AND + ORDER BY query |
|---|
| Problem with INNODB -> BETWEEN AND + ORDER BY query [message #3749] |
Fri, 28 November 2008 05:47 |
Juergen Messages: 1 Registered: November 2008 |
Junior Member |
|
|
Hi there,
I have a performance problem with inno.
I switched one table from MyISAM to INNODB because of transaction support:
the following query
SELECT * FROM table
WHERE artnr BETWEEN '120000' AND '130000'
AND menge IS NOT NULL
ORDER BY art ASC, p DESC
takes 0.8s to load with myisam, which is ok.
But with inno it takes 120+ sec (with heavy I/O activity).
The table contains only 500000 records. In both cases, no key is used.
But even if I add a key to artnr, the results are the same.
In my eyes this should not happen, even with no keys.
Another thing I don't understand:
MyISAM uses the index, INNODB not:
+----+-------------+-------+------+---------------+------+---------+------+--------+-----------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+------+---------------+------+---------+------+--------+-----------------------------+
| 1 | SIMPLE | table | ALL | TEST | NULL | NULL | NULL | 533968 | Using where; Using filesort |
+----+-------------+-------+------+---------------+------+---------+------+--------+-----------------------------+
If I change '130000' to something between '120000' and '128982' (or add a force index) the key is used with INNO but the performance is still very poor.
+----+-------------+-------+-------+---------------+------+---------+------+-------+-----------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+-------+---------------+------+---------+------+-------+-----------------------------+
| 1 | SIMPLE | fpos | range | TEST | TEST | 48 | NULL | 86134 | Using where; Using filesort |
+----+-------------+-------+-------+---------------+------+---------+------+-------+-----------------------------+
Does anybody know, why INNODB is so slow with this query?
I mean, maybe I can add a lookup myisam table or add an efficient index, but I don't understand what is happening in the background. I even could export all data manually to a csv file, sort the result with excel and reimport it to the database in less time (The "SELECT * FROM TABLE" statement takes 2 second to dump all data).
Oh, I forgot: The table:
CREATE TABLE `tableX` (
`id` int(10) unsigned NOT NULL auto_increment,
`art` int(11) unsigned NOT NULL default '0',
`p` int(11) unsigned NOT NULL default '0',
`artnr` varchar(6) character set latin1 default NULL,
`name` varchar(45) character set latin1 default NULL,
...,
PRIMARY KEY USING BTREE (`id`),
KEY `TEST` USING BTREE (`artnr`)
) ENGINE=InnoDB/MyISAM
Server = 5.1
|
|
| | Topic: MYSQL DISTINCT |
|---|
| MYSQL DISTINCT [message #3747] |
Thu, 27 November 2008 02:17 |
daredavil82 Messages: 1 Registered: November 2008 Location: Malaysia |
Junior Member |
|
|
hello everyone...
i like to know what is a suitable mysql clause instead of DISTINCT clause? i mean which clause same meaning and can produce same result as DISTINCT clause? how about GROUP BY? can i use GROUP by instead of DISTINCT clause?
|
|
| | Topic: Possible memory leak in 5.0.67 b7 |
|---|
| Possible memory leak in 5.0.67 b7 [message #3744] |
Wed, 26 November 2008 17:32 |
avinson Messages: 1 Registered: November 2008 Location: United States |
Junior Member |
|
|
I run a MySQL cluster with a single master and multiple slaves. All systems are running MySQL enterprise 5.0.56. I decided to experiment with the latest percona release on one of the slaves to see if there was a significant difference in performance. What I'm seeing is that the mysqld process slowly grows in memory size until eventually the system starts swapping. I thought it was possible that the percona release used slightly more memory overall so I reduced innodb buffer pool from 24G to 20G but I am still seeing the same behavior. It looks like there may be a memory leak or there is a setting that I need that I don't have. Here are my relevant config options:
skip-locking
key_buffer_size = 256M
max_allowed_packet = 16M
table_cache = 256
sort_buffer_size = 8M
myisam_sort_buffer_size = 8M
innodb_buffer_pool_size = 20G
innodb_log_file_size = 256M
innodb_file_per_table = 1
innodb_thread_concurrency = 0
innodb_file_io_threads = 10
innodb_max_dirty_pages_pct = 70
innodb_flush_method = O_DIRECT
innodb_support_xa = 0
query_cache_type = 1
query_cache_size = 64M
long_query_time = 1
max_connections = 2048
thread_cache_size = 32
With 24G innodb buffer pool size on the 5.0.56 enterprise binary the mysqld process is stable at 27G resident size in memory. The 5.0.67 percona binary steadily grows until it overwhelms the system. Is there a setting I'm missing? Any help is appreciated.
-Aaron
|
|
| | Topic: Avoided filesort, but more rows to exmaine, need advice! |
|---|
| Avoided filesort, but more rows to exmaine, need advice! [message #3743] |
Wed, 26 November 2008 16:54 |
matthew016 Messages: 4 Registered: November 2008 Location: Brussels |
Junior Member |
|
|
Hi,
I am pretty new in optimizing tables with index and may need some help.
This is my query:
EXPLAIN SELECT timestamp
FROM Meting_INT_COPY
WHERE blockid = '200811252000'
ORDER BY timestamp DESC
LIMIT 1
If I have an index(blockid),
EXPLAIN will return the following information:
type possible_keys key rows Extra
ref index_blockid index_blockid 2638 Using where; Using filesort
If I add an index(blockid,timestamp)
EXPLAIN will display the following:
type possible_keys key
rows Extra
ref index_blockid,index_blockid_timestamp index_blockid_timestamp 8248
Using where; Using index
The index(blockid,timestamp) avoid the filesort + returns the result from index ! (Using where; Using index)
But why for the index(blockid) 2638 rows are returned to examine and for a more
specific index(blockid,timestamp) 8248 rows are returned ?
Thank you very much for any answer !
|
|
| | Topic: MySQL functions |
|---|
| MySQL functions [message #3723] |
Thu, 20 November 2008 05:06 |
aloxy Messages: 1 Registered: November 2008 Location: Uganda |
Junior Member |
|
|
who can bail me out with a code hat produces a result to the following question?
Return a list of all flights along with their current un-discounted price
according to these tables;
CREATE TABLE `fly-four-less`.`flight_profiles` (
`FLP_ID` decimal(8,0) NOT NULL default '0',
`FLT_NUMB` decimal(4,0) default NULL,
`SEATS` int(11) default NULL,
`PRICE_CODE` varchar(50) default NULL,
`from` varchar(50) default NULL,
`to` varchar(50) default NULL,
`MON` varchar(1) default NULL,
`TUE` varchar(1) default NULL,
`NEXT_DAY` int(11) default NULL,
`WED` varchar(1) default NULL,
`THU` varchar(1) default NULL,
`FRI` varchar(1) default NULL,
`SAT` varchar(1) default NULL,
`SUN` varchar(1) default NULL,
`depart` datetime default NULL,
`arrive` datetime default NULL,
PRIMARY KEY (`FLP_ID`),
UNIQUE KEY `FLT_NUMB` (`FLT_NUMB`),
KEY `FK_flight_profiles_1` (`from`),
KEY `FK_flight_profiles_2` (`to`),
CONSTRAINT `flight_profiles_ibfk_1` FOREIGN KEY (`from`) REFERENCES `airports` (`IATA_CODE`),
CONSTRAINT `flight_profiles_ibfk_2` FOREIGN KEY (`to`) REFERENCES `airports` (`IATA_CODE`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
CREATE TABLE `fly-four-less`.`price` (
`PC_ID` int(11) NOT NULL default '0',
`PC_LEVEL` int(11) default NULL,
`PRICE` int(11) default NULL,
`FROM_DATE` datetime default NULL,
`TO_DATE` datetime default NULL,
PRIMARY KEY (`PC_ID`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
CREATE TABLE `fly-four-less`.`discount` (
`DIS_ID` int(11) NOT NULL default '0',
`DAYS_ADV_MIN` int(11) default NULL,
`DAYS_ADV_MAX` int(11) default NULL,
`PASSENGER_TYPE` int(11) default NULL,
`DISCOUNT` int(11) default NULL,
PRIMARY KEY (`DIS_ID`),
KEY `FK_discount_1` (`PASSENGER_TYPE`),
CONSTRAINT `discount_ibfk_1` FOREIGN KEY (`PASSENGER_TYPE`) REFERENCES `person_type` (`PT_ID`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
i came up with the following lines of query, but its not the results, if help avails can i have a deeper explanation on how the query digs to the table(s)?
SELECT * FROM FLIGHT_PROFILES F
WHERE F.NEXT_DAY=0;
Aloys
|
|
| | Topic: How to retrieve Images from Outlook contacts(PST file) ? |
|---|
| How to retrieve Images from Outlook contacts(PST file) ? [message #3720] |
Wed, 19 November 2008 01:42 |
simjith Messages: 3 Registered: November 2008 Location: Qatar |
Junior Member |
|
|
Hi all,
I am a newbie to PHP + MySQL.
Now I am on a project in which I have to import Outlook contacts(PST file) to MySQL database and must provide a web interface for the same similar to Outlook Contact interface. By now I had completed the project except the importing of profile images and attached images.
If anyone of you have any idea,regarding how to extract the profile images and attached images from Outlook contacts PST file to MySQL/access/excel or any thing else, please share your idea
Thanx in advance
Simjith
|
|
| | Topic: MySQL 5.1 version is stable in production environment |
|---|
| MySQL 5.1 version is stable in production environment [message #3719] |
Tue, 18 November 2008 04:15 |
bigrio Messages: 6 Registered: January 2008 Location: Beijing China |
Junior Member |
|
|
I am considering to use partitioning function provided in MySQL 5.1 version.
But 5.1 is development version according to my check on mysql site.
5.0 is current production version.
Can I use 5.1 in production environment without serious bugs , problems?
Thanks for your opinion.
|
|
| | Topic: Memcached Replication and Namespaces using MySQL Infrastructure. |
|---|
| Memcached Replication and Namespaces using MySQL Infrastructure. [message #3717] |
Sat, 15 November 2008 13:07 |
golanzakai Messages: 1 Registered: November 2008 Location: Israel |
Junior Member |
|
|
I would like to share with you guys a configuration method for memcached replication using mysql engine.
I wrote a blog entry describing it, I wish to hear feedback about the concept and if someone is already implemented such configuration on production environment.
http://golanzakai.blogspot.com/2008/11/memcached-replication -and-namespaces.html
Thanks
Golan Zakai
[Updated on: Sat, 15 November 2008 13:22]
|
|
| | Topic: Index of CHAR(32) vs BINARY(16) vs TEXT |
|---|
| Index of CHAR(32) vs BINARY(16) vs TEXT [message #3715] |
Thu, 13 November 2008 08:51 |
jedwood Messages: 1 Registered: November 2008 |
Junior Member |
|
|
Things I don't care about:
- storage space (rows in the thousands, not millions)
- security (this is not a password)
Things I do care about:
- speed
So, I have a variable length bit of text that can be as small as 3 characters but will never be longer than what the TEXT type allows. And I need to make a comparison against this field. My idea is to md5 it in PHP and store it as either CHAR(32) or BINARY(16) (if I use the raw_output = TRUE parameter). But I could also skip the hash step and just store it in TEXT, though that seems like it'd be a much poorer index than one of the first two options.
Any thoughts?
Thanks!
|
|
| | Topic: Maximum writes with MySQL |
|---|
| Maximum writes with MySQL [message #3713] |
Wed, 12 November 2008 11:04 |
v0idnull Messages: 1 Registered: November 2008 |
Junior Member |
|
|
I'm thinking of installing Piwik, an open source alternative to Google Analytics. It'll be used on popular websites, sites that could potentially generate millions of hits a day.
Piwik stores its information into a MySQL database. So I'm wondering, how stable would MySQL be doing millions upon millions of writes a day? Size of the database I'm not too concerned about. Everyday reports will be generated and the data flushed, but during the day the db will be hit hard with writes.
Anyone have architectural choices? Different db engine for mysql? Maybe MySQL isn't up to this task?
|
|
| | Topic: How to store temporary results best? |
|---|
| How to store temporary results best? [message #3712] |
Tue, 11 November 2008 11:43 |
Nyze1984 Messages: 1 Registered: November 2008 |
Junior Member |
|
|
Dear Community
I need your advice In our company we have a database with lots of tables and each table contains more then 500.000 rows.
My Job is to create a backend interface where user can select data via a filter interface and then keeping the results in a new table for later use. So far so good but i run into the problem that handling this large data takes a lot of time.
For Example
CREATE TABLE `searcg_xxtemp` (
`id` bigint(20) unsigned NOT NULL auto_increment,
`name` varchar(30) default NULL,
`pw` varchar(30) default NULL,
`email` varchar(255) default NULL,
`isanonym` tinyint(4) NOT NULL default '0',
`lastlogindatum` datetime NOT NULL default '0000-00-00 00:00:00',
PRIMARY KEY (`id`),
UNIQUE KEY `name` (`name`),
KEY `user_idkennung` (`id`,`kennung`),
FULLTEXT KEY `pw` (`pw`)
) ENGINE=MyISAM SELECT * FROM user WHERE {Filterdata goes here}
This takes a lot of time because in worst case the SELECT statement returns 500.000 rows. How can handle this the best and fastest way. I need a good solution how to store the temporary results for later use! Maybe you can give me a good hint
|
|
| | Topic: Performance: Indexing double join with subselect |
|---|
| Performance: Indexing double join with subselect [message #3711] |
Tue, 11 November 2008 10:59 |
Dr.Ro.Se Messages: 1 Registered: November 2008 |
Junior Member |
|
|
UPDATE: SELECT statement changed (the first one was not wrong but foolish...
I have the following table of objects identified by columns 'objID' and 'stackVer' with data columns data1, data2 ... (much more) in a table "Object".
Data processing must be made with the data stored in data1, data2..., using a certain set of parameters defined in table "Recipe".
For processing, different configurations may be used, defined in table "Config", each configuration holds the "recipeID" of the parameter set that is currently active.
Now I need all rows from the "Object" table that have value 'prepared' set in column "processingStatus" in table "ObjectStatus", and the appropriate recipeID from table "Config" with "configType" set to e.g. 'Auto'.
Output should be like this:
objID stackVer data1 data2 data3 recipeID
---------------------------------------------------
10001 1 66.4 2.3 3 101
I hope this is understandable 
The SELECT statement below works, but it [...] is quite slow (several seconds with 1.4 million objects, there will be several billions later...)
Any idea how I can speed up this query with appropriate indexing and/or a more sophisticated SELECT statement and/or better table organizing?
Many thanks for help!
Robert
Tables:
drop table if exists Object;
create table Object
(
objID bigint not null,
stackVer tinyint not null,
data1 real,
data2 real,
data3 real
);
create index idx_Object_objID on Object (objID);
create index idx_Object_stackVer on Object (stackVer);
create index idx_Object_objID_stackVer on Object (objID, stackVer);
drop table if exists ObjectStatus;
create table ObjectStatus
(
objID bigint not null,
stackVer tinyint not null,
configID tinyint,
processingStatus varchar(20) /* may be 'prepared', 'processing', 'processed' */
);
create index idx_ObjectStatus_objID on ObjectStatus (objID);
create index idx_ObjectStatus_stackVer on ObjectStatus (stackVer);
create index idx_ObjectStatus_objID_stackVer on ObjectStatus (objID, stackVer);
create index idx_ObjectStatus_processingStatus on ObjectStatus (processingStatus);
drop table if exists Config;
create table Config
(
configID tinyint not null,
configType varchar(20), /* may be 'Auto', 'Manual', 'User1', 'User2'... */
recipeID tinyint,
configDescription varchar(256)
);
insert into Config values (101, "Auto", 101, "Default");
insert into Config values (102, "Manual", 101, "Default");
drop table if exists Recipe;
create table Recipe
(
recipeID tinyint not null,
recipePath varchar(256),
recipeDescription varchar(256)
);
insert into Recipe values (101, "/path/to/recipes/Default", "Default");
My current SELECT statement to do this (updated):
select Object.objID, Object.stackVer, Object.IcB1, Object.IcB2, Object.IcB3, a.recipeID from Object left outer join
(select Config.recipeID, ObjectStatus.objID, ObjectStatus.stackVer, ObjectStatus.processingStatus from ObjectStatus left outer join
Config on Config.configID=ObjectStatus.configID where ObjectStatus.processingStatus='prepared') a
on a.objID=Object.objID and a.stackVer=Object.stackVer where a.recipeID limit 2;
EXPLAIN says (updated):
+----+-------------+--------------+------+----------------------------------------------------------------+-----------------------------------+---------+--------------------+---------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+--------------+------+----------------------------------------------------------------+-----------------------------------+---------+--------------------+---------+-------------+
| 1 | PRIMARY | <derived2> | ALL | NULL | NULL | NULL | NULL | 1070000 | Using where |
| 1 | PRIMARY | Object | ref | idx_Object_objID,idx_Object_stackVer,idx_Object_objID_stackVer | idx_Object_objID_stackVer | 9 | a.objID,a.stackVer | 11 | |
| 2 | DERIVED | ObjectStatus | ref | idx_ObjectStatus_processingStatus | idx_ObjectStatus_processingStatus | 23 | | 1176300 | Using where |
| 2 | DERIVED | Config | ALL | NULL | NULL | NULL | NULL | 2 | |
+----+-------------+--------------+------+----------------------------------------------------------------+-----------------------------------+---------+--------------------+---------+-------------+
[Updated on: Tue, 11 November 2008 12:49]
|
|
| | Topic: Self-join and sorting |
|---|
| Self-join and sorting [message #3709] |
Tue, 11 November 2008 07:10 |
neilf Messages: 2 Registered: October 2008 |
Junior Member |
|
|
Hi all.
I'm wondering if anyone has a solution for a problem I'm facing. I'm giving a simplified example for clarity (I hope), but I think it illustrates the problem.
Let's say I have a table called "Things" as follows:
Things
------
ThingID BIGINT
Timestamp BIGINT
Attribute VARCHAR(20)
And there are two indexes on this table as follows:
Indexes
-----
AttributeTimestampIndex (Attribute, Timestamp)
ThingIDIndex (ThingID)
My table is denormalised, so each ThingID can have multiple Attributes for each timestamp, all stored in different rows.
However, importantly, each ThingID cannot have multiple timestamps.
For example:
Thing ID | Timestamp | Attribute
1 | 1 | 'foo'
1 | 1 | 'bar'
2 | 1 | 'foo'
2 | 1 | 'baz'
3 | 2 | 'foo'
Let's say I want to find the timestamp of all Things that have Attributes of 'bar' and 'foo', I execute the following query:
SELECT t1.Timestamp FROM Things t1, Things t2 WHERE t1.AttributeA = 'foo' AND t2.ThingID = t1.ThingID AND t2.AttributeA = 'bar' ORDER BY Timestamp
The query optimizer (correctly) figures out that 'bar' is a more selective attribute, and evaluates t1 first, then joins to t2 on the ThingIDIndex index.
The problem is, even though I have Timestamp in my index (which it should hopefully be able to use for sorting), MySQL ends up doing a Filesort, because I am selecting t1.Timestamp rather than t2.Timestamp (even though they are always the same, since there can't be multiple Timestamps for a ThingID).
The problem is I don't know in advance whether t1 or t2 will have the more selective clause applied to it, so I don't know whether to select from t1 or t2 in advance.
If I remove the t1 alias from Timestamp when I select it, then MySQL complains about an ambiguous column name (of course).
I can add the following clause to my SQL to enforce the fact that t1 and t2 have the same Timestamp.
t2.Timestamp = t1.Timestamp
But the effect is the same: MySQL still complains about an ambiguous column, and I still have to choose one of t1 or t2 to select from, thus I still have a 50% chance of MySQL having to do an unnecessary Filesort.
I'm not very keen on having to do a Filesort for a couple of reasons: 1. it's unnecessary work, and, more importantly 2. I can't start streaming the results to the client early, since I have to wait for the entire query to complete before I can get any results.
Anybody got any ideas as to how to solve this?
|
|
| | Topic: SHOW PROCESSLIST not showing all users? |
|---|
| SHOW PROCESSLIST not showing all users? [message #3707] |
Mon, 10 November 2008 14:34 |
amandabee Messages: 1 Registered: November 2008 Location: New York, NY |
Junior Member |
|
|
I'm trying to troubleshoot our mysql server, which keeps maxing out its connections. However, I can't get a full processlist for all users.
I'm logged in as root, and elsewhere as a client that has "process_priv" granted, but when I run "show processlist" I only see the list for the current user.
Am I missing something? All my reads of the documentation http://dev.mysql.com/doc/refman/5.0/en/show-processlist.html suggest that I ought to be able to see every process.
|
|
| | Topic: MySQL crashes system |
|---|
| MySQL crashes system [message #3706] |
Mon, 10 November 2008 14:22 |
mani Messages: 1 Registered: November 2008 |
Junior Member |
|
|
Hi,
Environment:
CentOS 3.5 (2.4.21)
MySQL 5.0.64 Enterprise Server (Commercial)
2 GB RAM, 1 CPU
System variables:
bulk_insert_buffer_size 8388608
key_buffer_size 268435456
myisam_data_pointer_size 6
myisam_max_sort_file_size 2146435072
myisam_sort_buffer_size 67108864
We have an application which pumps in excess of 500 records/sec to MYISAM database and after 10 hrs or so, system silently crashes.After doing a power cycle, it is found the database has crashed and tables are corrupted.
<snip>
mysql> select * from xyz;
ERROR 1194 (HY000): Table 'xyz' is marked as crashed and should be repaired
<snip>
Also, we saw high iowait time (90%) for 1 min in every 3 min cycle.
Has anyone seen this problem before?
Thanks,
Mani.
|
|
| | Topic: Change from MyISAM to InnoDB? |
|---|
| Change from MyISAM to InnoDB? [message #3704] |
Fri, 07 November 2008 08:58 |
armakuni Messages: 3 Registered: July 2008 |
Junior Member |
|
|
Usually people are talking about big databases but we have a very small database (~40MB, ~20 tables) where all tables are using MyISAM engine. My beginner question is: does it make sense to change to InnoDB engine? Current ratio between writes and reads from mysqlreport (last 80 days):
__ Questions ___________________________________________________________
Total 891.67M 128.0/s
DMS 685.86M 98.4/s %Total: 76.92
Com_ 111.21M 16.0/s 12.47
COM_QUIT 97.60M 14.0/s 10.95
-Unknown 3.00M 0.4/s 0.34
Slow 10 s 1 0.0/s 0.00 %DMS: 0.00 Log: OFF
DMS 685.86M 98.4/s 76.92
SELECT 373.76M 53.6/s 41.92 54.50
REPLACE 202.12M 29.0/s 22.67 29.47
UPDATE 107.40M 15.4/s 12.04 15.66
DELETE 2.42M 0.3/s 0.27 0.35
INSERT 169.72k 0.0/s 0.02 0.02
Com_ 111.21M 16.0/s 12.47
set_option 102.20M 14.7/s 11.46
stmt_prepar 2.99M 0.4/s 0.34
stmt_execut 2.99M 0.4/s 0.34
and table lock status:
__ Table Locks _________________________________________________________
Waited 9.49M 1.4/s %Total: 0.79
Immediate 1.19G 170.1/s
We are only interested about performance (speed) and reliability is not so important. Because there are lot of read and writes InnoDB should in theory be good if I understand correctly differences between MyISAM and InnoDB. But does the engine really matter in such a small database?
|
|
| | Topic: Optimizing Query |
|---|
| Optimizing Query [message #3703] |
Fri, 07 November 2008 08:18 |
USAF_Pride Messages: 1 Registered: November 2008 |
Junior Member |
|
|
I have a database with 9 tables of unique data and 1 table that links it all together. I know the speed issue is all the JOINS and I am way outside of my league in attempting to fix it. The one table that links everything together only stores the primary ID from the other 9 tables. Here is the sql
SELECT DISTINCT guid, name as unique_name, ip as unique_IP, Violation, vdate, vtime
FROM tbl_player_kick
LEFT JOIN tbl_guid ON tbl_player_kick.guid_id = tbl_guid.guid_id
LEFT JOIN tbl_kick ON tbl_player_kick.kick_id = tbl_kick.kick_id
LEFT JOIN tbl_name ON tbl_player_kick.name_id = tbl_name.name_id
LEFT JOIN tbl_ip ON tbl_player_kick.ip_id = tbl_ip.ip_id
LEFT JOIN tbl_violation ON tbl_player_kick.viol_type = tbl_violation.violation_id
LEFT JOIN tbl_date ON tbl_player_kick.date_id = tbl_date.date_id
LEFT JOIN tbl_time ON tbl_player_kick.time_id = tbl_time.time_id
WHERE tbl_player_kick.banned = 1
and the explain that goes with it
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE tbl_player_kick ref banned banned 2 const 262650 Using where; Using temporary
1 SIMPLE tbl_guid eq_ref PRIMARY PRIMARY 4 aaser2_bansdbase.tbl_player_kick.guid_id 1
1 SIMPLE tbl_kick eq_ref PRIMARY PRIMARY 8 aaser2_bansdbase.tbl_player_kick.kick_id 1 Using index
1 SIMPLE tbl_name eq_ref PRIMARY PRIMARY 4 aaser2_bansdbase.tbl_player_kick.name_id 1
1 SIMPLE tbl_ip eq_ref PRIMARY PRIMARY 4 aaser2_bansdbase.tbl_player_kick.ip_id 1
1 SIMPLE tbl_violation eq_ref PRIMARY PRIMARY 8 aaser2_bansdbase.tbl_player_kick.viol_type 1
1 SIMPLE tbl_date eq_ref PRIMARY PRIMARY 4 aaser2_bansdbase.tbl_player_kick.date_id 1
1 SIMPLE tbl_time eq_ref PRIMARY PRIMARY 8 aaser2_bansdbase.tbl_player_kick.time_id 1
Any help would be greatly appreciated.
[Updated on: Fri, 07 November 2008 09:28]
|
|
| | Topic: Need Help for Mysql-Query optimization |
|---|
| Need Help for Mysql-Query optimization [message #3702] |
Fri, 07 November 2008 07:26 |
yashvant Messages: 1 Registered: November 2008 |
Junior Member |
|
|
I have run this query make a more time please any body optimize this query as blew
$sql="SELECT f.feeder_code, f.subdivision_code FROM `feeder` f, sentout s
where ( f.start_date <= '2008-07-01' and f.end_date >= '2008-07-01' and
(s.month = 7 and s.year = 8 )) and f.feeder_code = s.feeder_code
UNION
SELECT feeder_code, subdivision_code FROM ht WHERE ht.month = 7 and
ht.year = 8 and concat( feeder_code, subdivision_code ) NOT IN
(SELECT concat( f.feeder_code, f.subdivision_code )
FROM `feeder` f, sentout s where (f.start_date <= '2008-07-01' and f.end_date >='2008-07-01' and (s.month = 7 and s.year = 8)) and f.feeder_code = s.feeder_code)
UNION
SELECT feeder_code, subdivision_code FROM lt WHERE lt.month = 7
and lt.year = 8 and concat( feeder_code, subdivision_code ) NOT IN
( SELECT concat( f.feeder_code, f.subdivision_code ) FROM feeder f, sentout s
where (f.start_date<= '2008-07-01' and f.end_date >= '2008-07-01' and
(s.month = 7 and s.year = 8)) and f.feeder_code = s.feeder_code
) AND concat( feeder_code, subdivision_code ) NOT IN ( SELECT concat( feeder_Code, subdivision_code )
FROM ht WHERE ht.month = 7 and ht.year = 8)
ORDER BY feeder_code, subdivision_code";
solved this problem because this take a more time
[Updated on: Fri, 07 November 2008 07:31]
|
|
| | Topic: Optimizing simple order by query |
|---|
| Optimizing simple order by query [message #3686] |
Mon, 03 November 2008 18:53 |
novice.program Messages: 1 Registered: November 2008 |
Junior Member |
|
|
Hello I am having problems in optimizing a simple order by clause.
the query is:
explain SELECT `Entity`.`title`, `Entity`.`description`, `Entity`.`user_id`, `Entity`.`thumb_url`, `Entity`.`entity_uid`, `Entity`.`num_fachaks`, `Entity`.`number_of_views` FROM `entities` AS `Entity` ORDER BY `Entity`.`created` DESC LIMIT 21 \G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: Entity
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 11
Extra: Using filesort
I have index on the created filed by really dont understand that why mysql does not use it and prefers file sort.. the created field is a timestamp default current_timestamp.
Please help ... i am new to mysql and badly stuck.
Thanks.
|
|
| | Topic: Increasing performance by parallel processing or multiple mysqld |
|---|
| Increasing performance by parallel processing or multiple mysqld [message #3685] |
Sat, 01 November 2008 06:18 |
nook Messages: 1 Registered: November 2008 |
Junior Member |
|
|
I'm running a search engine for realestate and I am in the process of acquiring a new server.
My search engine process is very well suited for parallelization, therefore I want to know how I can tweak mysql to use this in an optimal way.
I've noticed that on my present server mysql only uses one processor, the other is not used at all....
I see a possible number of possibilities:
1) putting different databases on different hard drives so read/write can be made fully parallel.
2) for each database running a different mysqld on its own port.
3) configuring my linux server so that it consist of (say) 4 virtual linux servers. Then on each of these 4 servers I install mysql for a different database.
I haven't been able to find any references on this subject.
|
|
| | Topic: Choosing a server |
|---|
| Choosing a server [message #3679] |
Wed, 29 October 2008 19:56 |
carrigan Messages: 1 Registered: October 2008 |
Junior Member |
|
|
Hi guys,
I'm choosing a new server for the upcoming dedicated mysql server.
Few questions:
1) SCSI 15k, SCSI 10k, Raptor 10k in RAID 10?
2) Server options:
a) AMD Opteron 870, 8x2.0GhZ, 1000Mhz CPU, 8x1MB CPU Cache, DDR400 ECC Memory
b) Intel Xeon 5345, 8x2.33GhZ, 1333Mhz CPU, 2x8MB CPU Cache, FB-DIMM 533/677 Memory
I know it varies from application to application but let's assume for a general high load mysql server.
Thanks!
|
|
| | Topic: PRIMARY key not working |
|---|
| PRIMARY key not working [message #3677] |
Wed, 29 October 2008 12:29 |
jeffStrahl Messages: 1 Registered: October 2008 |
Junior Member |
|
|
Hello,
I have a query that is joining multiple tables. I have it optimized rather well, except for one table. I can't for the life of me figure out why that table is not using the pk as the key. It is causing the query to filesort and use a temp table, changing the query time from 0.003s to 23s, obviously too long.
Attached is a file with the database and query details.
Any assistance is greatly appreciated, for I don't know what it could be.
|
|
| | Topic: Limits and datatype for connection_id() |
|---|
| Limits and datatype for connection_id() [message #3650] |
Fri, 24 October 2008 09:25 |
Poul Messages: 1 Registered: October 2008 |
Junior Member |
|
|
Hi,
we use connection_id() return value to fill some fields in a mysql-myisam database.
I have spent a lot of time researching about the returned value, but I have not found any interesting information, so I'm asking this blog which I follow from one year now.
My questions are:
- select connection_id() returns an integer value: what are the limits? How should a declare the field to store it?
- what will happen when the limits are reached? Yes, it is possible; we are managing millions of connections per day on a single database...
- is it there any other better parameter available in a query, to ensure that two concurrent users will have returned different values?
Many thanks in advance.
|
|
| | Topic: One to many -- Select 3 subitems for every item. |
|---|
| One to many -- Select 3 subitems for every item. [message #3649] |
Thu, 23 October 2008 09:12 |
wesleyh Messages: 1 Registered: October 2008 |
Junior Member |
|
|
Hello,
I have two tables with a one to many relationship:
collection: id, user_id, title, description, item_count
items: id, collection_id, title, description
Now, I want to display the collections on a result page. (paginated). But, for each collection, I want to show the first 3 or so items that are in it.
What is the best possible way to do this efficiently? If I just do a straight up join I get the collection information 3 times as well, which I do not need.
Or should I just do a second query after having queried for the collections first? (Then using the collection_ids to get the items)?
What I do now:
SELECT c.id, c.title, c.description, GROUP_CONCAT(i.id) AS item_ids FROM collections c INNER JOIN items i ON i.collection_id = c.id ORDER by c.created_at DESC LIMIT 10
Then i collect those item_ids in php and do a "WHERE id IN (..)" on all those ids.
But I need to know if there is a better solution.
Thank you for your help,
Wesley
|
|
| | Topic: slave sql thread always at 2 seconds |
|---|
| slave sql thread always at 2 seconds [message #3648] |
Thu, 23 October 2008 03:47 |
mancdaz Messages: 5 Registered: March 2008 |
Junior Member |
|
|
How come the slave sql thread always shows a time of not less than 2:
30884 system user NULL Connect 2 Has read all relay log; waiting for the slave I/O thread to update it NULL
?
Daz
|
|
| | Topic: Avoiding Stored Procedure Metadata Requests |
|---|
| Avoiding Stored Procedure Metadata Requests [message #3645] |
Wed, 22 October 2008 06:09 |
shlomoid Messages: 10 Registered: September 2007 |
Junior Member |
|
|
Hi all,
Do you how to prevent the mysql connector/net asking the server for the stored procedure metadata each first execution? I can't seem to find any place that explains how to do that properly.
I've tried providing as much information about the SP parameters as possible, but nothing works.
Here is an example:
When launching a stored procedure from .net, I see this in the general log:
22 Init DB db_name
22 Query SELECT * FROM INFORMATION_SCHEMA.ROUTINES WHERE OUTINE_SCHEMA='db_name' AND ROUTINE_NAME='mySP'
22 Query SHOW CREATE PROCEDURE `db_name`.`mySP`
22 Query call mySP (10)
It makes sense to get the stored procedure code if the DAL needs to know the data types of the parameters, but it does that even when I am strong-typing the variables as much as I can.
In this case:
MySqlParameter param1 = new MySqlParameter("?p_param", MySqlDbType.Int32);
param1.Value = 10;
param1.Direction = ParameterDirection.Input;
param1.IsNullable = false;
param1.MySqlDbType = MySqlDbType.Int32;
cmd.Parameters.Add(param1);
cmd.ExecuteNonQuery();
Is there a correct way to do this that avoids those queries?
Thanks in advance!
Shlomo
|
|
| | Topic: Optimization: Only InnobDB, no MyISAM |
|---|
| Optimization: Only InnobDB, no MyISAM [message #3636] |
Sat, 18 October 2008 18:06 |
colnector Messages: 1 Registered: October 2008 |
Junior Member |
|
|
As I've chosen to use only InnoDB for my tables, I wish to know which configuration values I may cut to save on memory.
Also, I understand that MyISAM is used anyway for temporary tables. It also seems like the default "MySQL" and "Information_Schema" databases use MyISAM. Can this be changed or am I stuck with MyISAM anyway?
Thanks
Colnect Collectors Community
|
|
| | Topic: Max_data_length is Zero |
|---|
Max_data_length is Zero [message #3633] |
Thu, 16 October 2008 03:03 |
cowboymathu Messages: 7 Registered: October 2008 |
Junior Member |
|
|
Hi All,
show table status\G
*************************** 1. row ***************************
Name: name_table
Engine: MyISAM
Version: 10
Row_format: Dynamic
Rows: 259956366
Avg_row_length: 128
Data_length: 33460714548
Max_data_length: 0
Index_length: 44122680320
Data_free: 3640
Auto_increment: NULL
Create_time: 2008-10-06 14:59:57
Update_time: 2008-10-16 12:19:30
Check_time: 2008-10-06 15:32:38
Collation: latin1_swedish_ci
Checksum: NULL
Create_options: partitioned
Comment:
I have almost 260 millions of records with 73 GB disk capacity. What does actually "Max_data_length is zero" mean? How can I increase that?
Thanks,
[Updated on: Thu, 16 October 2008 03:03]
|
|
| | Topic: Loading large tables with Unique Keys |
|---|
| Loading large tables with Unique Keys [message #3632] |
Wed, 15 October 2008 16:59 |
johndz Messages: 1 Registered: September 2008 Location: Broomfield, Colorado |
Junior Member |
|
|
This subject resurrects the trick shown in this thread for loading or altering large tables.
Alter table performance with MyISAM
I am struggling with some random cases where the REPAIR TABLE process is showing "Repair with keycache" instead of "Repair by sorting". In my latest test, on try # 2 it is "Repair by Sorting" which I believe is desired. Note that I use the "QUICK" option of the REPAIR TABLE command.
What would cause this to go into "Repair with keycache"?
Thanks,
johndz
|
|
| | Topic: nothing |
|---|
| nothing [message #3630] |
Tue, 14 October 2008 11:05 |
chocobn69 Messages: 1 Registered: October 2008 |
Junior Member |
|
|
|
No Message Body
[Updated on: Thu, 30 October 2008 06:34]
|
|
| | Topic: Table data covered entirely by indexes |
|---|
| Table data covered entirely by indexes [message #3629] |
Tue, 14 October 2008 10:48 |
neilf Messages: 2 Registered: October 2008 |
Junior Member |
|
|
Hi all.
I have a table that is entirely covered by indexes. That is, every query that I want to perform on the table can be satisfied entirely from indexes, without having to read the table data at all.
This may seem like a bizarre thing to want to do, but does anyone know if it's possible to store the data in MySQL in *just* the indexes, without having to store the table data at all, since the table data is never actually accessed?
If this isn't possible in MySQL, does anyone know of any alternatives? I know that certain column-based stores (for example, C-store) might allow this, by storing my indexes as a set of projections over the data.
Thanks.
Neil.
|
|
| | Topic: mysql performance on IBM Power6 systems |
|---|
| mysql performance on IBM Power6 systems [message #3624] |
Mon, 13 October 2008 15:44 |
fgarat Messages: 1 Registered: October 2008 Location: Montevideo - Uruguay |
Junior Member |
|
|
Hi Peter and everyone,
im starting to evaluate the hadware plataform for a mysql database and i start wandering about how mysql perform on IBM new Power 6 systems as they are becaming more cheap and have the fatest clock per core.. so..
anyone have some info on this?.
AIX and mysql? Linux and mysql? mysql and power6?.
Thanks all,
Facundo.-
pd: sorry my bad english
|
|
| | Topic: sysbench |
|---|
| sysbench [message #3616] |
Wed, 08 October 2008 12:38 |
noah Messages: 1 Registered: October 2008 |
Junior Member |
|
|
I'd like to ask you guys something about sysbench, particularly about the OLTP complex test.
It seems that the test exercises deadlocks by design. The way the transactions are built seems to favor the occurrence of deadlocks, please correct me if I'm wrong.
So far I have made tests with MySQL and PostgreSQL, since both have MVCC the deadlocks didn't arise. Is there a way to "disable" MVCC on MySQL? my idea is to evaluate the performance of MVCC versus locking system.
Thanks!
-Noah
|
|
| | Topic: Comparing two query results |
|---|
| Comparing two query results [message #3609] |
Fri, 03 October 2008 16:38 |
mousey182 Messages: 1 Registered: October 2008 |
Junior Member |
|
|
Hey Guys!
My first time here so hello all and thanks in advance for any help you may be able to give me.
I have a table called links with the following structure:
| link_id | type | frequency |
The type can be either link or advert.
I also have a table called links to keywords with the following structure:
| s_link_id | d_key_id |
I have a query that finds the most popular keywords by finding the keywords associated with the most frequently viewed links of type link.
I'm now trying to find the links of type advert that have keywords associated with them that match most closely the keywords of the links.
Any ideas how this can be achieved?
Ive tried the Like expression, but if the keywords associated with each type of link dont match exactlty then it doesnt return any results.
Thanks guys, I hope this makes sense!
Matt
|
|
| | Topic: Memory tables in a separate db |
|---|
| Memory tables in a separate db [message #3604] |
Thu, 02 October 2008 10:25 |
mylesmg Messages: 1 Registered: October 2008 |
Junior Member |
|
|
I have a medium-sized database with about 100M records in about 300 tables. The largest tables have at more 9M records. To help speed up some of the queries, which need to display real-time results, I have the system creating MEMORY tables (TEMPORARY tables didn't work for me). I have about 7K MEMORY tables, and I was wondering if putting these temporary tables into a separate database schema would make a performance difference.
Currently I'm performing an average of 125 queries per second. The entire database fits into memory (INNODB buffer pool) and seems to perform quite well overall.
My concern is that when I have to start using replication I may encounter issues with these temporary tables. MySQL Cluster is not available as an option on Windows yet, and since that's all I know how to administer I have to keep using it.
So what do we know about the performance of tables being accessed across databases and the performance impact of this?
Thanks in advance,
Michael Myles
www.aimcrm.com
|
|
|
Pages (6): [1 ]
Current Time: Tue Dec 2 16:19:58 EST 2008
Total time taken to generate the page: 0.58402 seconds |