61 questions
0
votes
0
answers
172
views
Importing binlog as an sql file into mysql database
I created an sql file from a binlog using this command:
mysqlbinlog --start-datetime="2024-02-22 00:00:00" --stop-datetime="2024-02-23 15:55:00" /var/lib/mysql/binlog.000096 > ./...
- 379
1
vote
1
answer
219
views
MySQL Binlog increases beyond normal specified
i have binlog enabled for mysql 5.7 and i am seeing that some files are created twice for the same time (one is 100 mb other is 2.3GB) they were not being done previously the bin log size is 100 mb as ...
- 11
0
votes
1
answer
471
views
Caused by: com.github.shyiko.mysql.binlog.event.deserialization.MissingTableMapEventException: No TableMapEventData has been found for table id:137
We have a simple spring boot 3.1 app that wraps debezium. We upgraded to the latest version 2.4.0.Final. Now getting this error after the service was running was running for a few hours:
2023-10-05 07:...
- 12.3k
0
votes
0
answers
30
views
How to notify row changes of rds mysql version 5.6.4?
I need to monitor row changes of 5 tables of an ancient MySQL instance - version 5.6.4.
I know that aurora has lambda_call function, so I could combine this function with triggers and get what I need. ...
- 168
0
votes
1
answer
47
views
cannot enable bin-logs for mysql
I have an old 5.7 mysql database that crashed last week. But i was able to start it again.
Now we have an issue where certain settings are disabled but were or are still enabled in my.cnf.
The setting ...
- 1
2
votes
1
answer
132
views
When are mysql save a new binlog file?
I'm developing 1 java application to reading a binlog file to catch query event. Everything would be fine without exception. But what will happen if my application was crashed, i will miss many event ...
- 53
0
votes
1
answer
506
views
Does Kafka support CDC from a source of MariaDB by binlog?
We have a production system running MariaDB, and hope to build an extract-transformation-load (ETL) module with near real-time changed data capture through the database's binary log (binlog) ...
- 2,123
1
vote
1
answer
1k
views
expire_log_days parameter value not updated?
I have one MySQL RDS and i am trying to change the expire_log_days parameter value, but i'm unable to change because it's unmodified parameter. But while i am checking inside database it shows 0, ...
- 73
0
votes
0
answers
389
views
Debezium MySQL Connector - Primary Server Configuration for binlog_row_image setting
We have a primary MySQL server and a secondary/replica server. Debezium connector is configured to read from the replica server.
Per the Debezium documentation, we configured the below settings on the ...
0
votes
1
answer
90
views
Mariadb binlog query event q_flag
I have a program which can read MySQL binlog. Recently I use the program to parse a binlog generated by MariaDb 10.3. I found that there are Q_FLAGS in the QUERY_EVENT which was not specified in the ...
- 127
0
votes
0
answers
23
views
What does generated by server on drop tables mean [duplicate]
Hi i create a new table using phpmyadmin create table not query and suddenly all my tables are recreated by itself and lost all data i tried to check last binlog and i found some strange lines like ...
1
vote
2
answers
1k
views
Can MySql binlog have more than one open transaction?
Can MySql binlog have more than one open transaction at the same time (= events of different transactions are interleaved in binlog) ?
There is XID event that contains transaction ID but there is no ...
- 30k
0
votes
2
answers
1k
views
MySql Binlog Update Statement Where Clause Has All Table Columns, Even If Client Uses Just Primary Key In Where Clause When Running Query
On Examination of BinLog Found Following Statement : It has All table columns in where clause. The query I passed to mysql had only 1 Column- Primary Key Column, then why the log wrote all columns in ...
- 23
0
votes
0
answers
144
views
Whether table names and db names are case sensitive in mysql binlog file
I create a table as follow:
create table INTERFACE_INFO_PK_SIMPLE
(
API_ID VARCHAR(128) not null primary key,
API_METHOD VARCHAR(24),
API_PATH VARCHAR(1024),
API_STATUS ...
- 764
0
votes
1
answer
213
views
MySql:If i set binlog format to row,dose change buffer still valid？
When i use "satement" binlog and non-unique index,mysql will use "change buffer" avoid reading from disk.
But if set binlog format to "row", binlog need the all columns ...
- 3
2
votes
0
answers
1k
views
Debezium reads binlog newest if offset storage is empty or just created&
I am running reading binlogs with Debezium, but when I start new reading thread it reads all create statements for table from the beginning, but I dont need them(op=c). I need to handle create/update/...
- 115
0
votes
1
answer
254
views
binlog not recording DML statement in mysqlcluster
I have created a MySQL NDB Cluster consisting of 2 data node, 1 mgm node and 1 mysqld for testing. Also i have created a slave for NDB to INNODB replication. My binlog are not recording DML statement ...
0
votes
2
answers
502
views
Bin Log (Tail Log) in Google Cloud Spanner
Can we get bin log of DML statements execution in Google Cloud Spanner.
Tried enabling "Data Access Write" Logs, but it does not give the same bin log as mysql or other relational databases.
https://...
- 361
0
votes
1
answer
188
views
MySQL bin log breaking in chunks
I am trying to delete records from a table like
DELETE * FROM table WHERE x=1 LIMIT 200000;
When I am checking my bin logs it is showing breaking of this delete statement into further smaller chunks ...
- 714
0
votes
1
answer
890
views
Where can I find a variable about max bin log files in Maria DB ( I aint mean to say max_binlog_size)
Too many binary log files are in the disk.
the number of log files keep 14~16 as below. (normally 14)
mysql-bin-changelog.028255 134223043
mysql-bin-changelog.028256 134228506
mysql-bin-...
- 57
1
vote
0
answers
161
views
How to let Debezium switch to next binlog automatically
In my project, Debezium still pointing on the same binlog, he doesn't switch to the next binlog. I have always to restart the connector debezium so he can read from the current binlog.
How can I let ...
- 65
1
vote
2
answers
2k
views
How to let debezium start reading binlog from the last row
I'm trying to let Debezium start reading the binlog from the bottom of the file directly.
Could someone help in this matter please ?
- 65
2
votes
0
answers
1k
views
mysql-binlog-connector-java - Could not find first log file name in binary log index file
I am doing a POC on this library, and I am having some difficulties setting it up. I have an extremly simple MySQL setup:
- a single VM (no master-slaves)
- centos 7
- MySQL version 5.7
Here is the ...
- 365
2
votes
3
answers
2k
views
Using python Django and Mysql. Data won't inserted into MySQL due to Binlog
When I use django to design a website and delopy, I find I have some trouble if I use MySQL that the Binlog is activate( format:STATEMENT).
There are my settings for this deployment:
Django based ...
- 21
6
votes
1
answer
963
views
Slow insert and update commands during mysql to redshift replication
I am trying to make a replication server from MySQL to redshift, for this, I am parsing the MySQL binlog. For initial replication, I am taking the dump of the mysql table, converting it into a CSV ...
0
votes
0
answers
739
views
What happens when mysql binlog reaches mysql-bin.999999
Lets say my mysql replication is enabled for long time, and the server has been quite busy, so new bin log files get created every few seconds. So in a few months, it is possible for latest bin log ...
- 1,514
2
votes
1
answer
66
views
Why mysqlbinlog utility displays two values instead of one?
I am trying investigate something in binlog by mysqlbinlog utility. When I add verbose parameter -vv I see INSERT query with this part
@2=-27170 (38366) /* SHORTINT meta=0 nullable=0 is_null=0 */
...
- 21
1
vote
1
answer
7k
views
How can i listen MySQL database changes in real time with Node.js or PHP
I need to create a system with multiple databases one of this is the master database this database need to replicate only structural changes to other databases for example:
When I register a new user ...
- 2,199
1
vote
0
answers
511
views
how to recover mysql data without binary logs
I wanna recover some data that were deleted by accident, but mysql binary log is off. no bin-log file for recovering the history data. is there any other approach to solve the problem ?
1
vote
0
answers
520
views
converting mysql binlogs to text format without using mysqlbinlog tool
I need to convert the mysql binlogs to text format in PHP script. I am aware of using mysqlbinlog tool to convert it to text. But I am not supposed to run any system commands(or any shell command) in ...
- 1,641
1
vote
2
answers
2k
views
Expire bin log in minutes
In the mysql conf file my.cnf, I want to add expiry time for the bin-logs.
After doing a quick research on that, I got to know how to set it for days.i.e., expire_logs_days=3.
I want to set the ...
- 915
0
votes
1
answer
3k
views
MySQL Slave replication failure: How to identify SQL which is causing it to fail
I am running MySQL on AWS RDS.. Slave replication is getting stuck because of following error. It says that it is not able to find a record in table of slave server... Is there a way to know which ...
- 10.9k
2
votes
1
answer
3k
views
MySQL - Find bin log position based on timestamp
I am using shyiko connector to stream bin log changes from a mysql cluster to downstream database systems.
A cluster = MySQL master + Primary slave + Secondary slave
When the listening MySQL system ...
- 1,316
1
vote
1
answer
3k
views
sync_binlog parameter
I was going through the documentation for sync_binlog parameter and found a discrepancy in sync_binlog parameter documentation.
The documentation here http://dev.mysql.com/doc/refman/5.6/en/...
- 173
0
votes
0
answers
436
views
Sphinx Searchd long to restart
we've been using Sphinx (2.0.4) for about 4 years, but recently (like a month ago), searches started having downtimes once an hour.
We have a big update of all the data during the night, which doesn'...
0
votes
1
answer
821
views
got "ClassNotFoundException" when reading binlog data from Kafka using Spark Streaming
I want to read the binlog data from kafka using Spark Streaming, the binlog data is collected using canal(which uses protobuf-2.4.1)，I have to use the protobuf-2.5.0 in the Spark Streaming environment....
- 71
0
votes
1
answer
528
views
Can MariaDB's Maxscale(BinLog Server) be used as a collector of binlogs from different mysql clusters?
I have read that Maxscale(BinLog Server) of MariaDB can be used for relaying the bin logs from a MySQL Cluster to a single BinLog Server, however i wanted to know if its possible to collect all the ...
- 403
3
votes
0
answers
853
views
MySQL: programatically get list of binlog files
How can I programatically get a list of a particular mysql instance's binlogs? ie. the same information provided by
mysql> show binary logs;
update: version 5.6 / AWS Aurora preferred.
- 304k
0
votes
1
answer
847
views
Does the MySql binlog contain transaction boundaries?
If I'm reading a MySql binlog, can I get an indication of which statements occur in the same transaction?
- 304k
0
votes
1
answer
71
views
mysql inside a transaction does the binlog guarantee the update oder of table
start transaction;
update a set b=1 where a=1;
update b set c=1 where a=1;
commit;
The binary log will record a, b update in order, but when would the b's update flush to log before than a?
how to ...
1
vote
1
answer
168
views
MySQL statement based replication: Does binlog contain exact queries executed on master?
We had to debug our web application for an event that occurred some days ago, and all we have are the MySQL replication binlogs (statement replication). Can we depend on those as being exactly the ...
1
vote
0
answers
269
views
mysql binlog output file is ambiguous?
I use
mysqlbinlog -f -v --base64-output=DECODE-ROWS mysql-bin.00001 > res.000001
my table has a column is varbinary. So res.000001 like this
### UPDATE db.tbl
### WHERE
### @1=967540772
### ...
- 2,926
0
votes
0
answers
446
views
binlog_do_db groups for mysql replication?
Is it possible to split the binlog_do_db to slave-groups?
I have a mysql master server with the databases "frontend, backend, events, lists".
Now I want to replicate the databses "events and lists" ...
3
votes
1
answer
9k
views
How to understand the timestamp in the mysql binlog?
the binlog always looks like this:
#140105 18:49:33 server id 2 end_log_pos 354 Query thread_id=1 exec_time=0 error_code=0
SET TIMESTAMP=1388918973/*!*/;
/* some SQL statement */
/*!*...
- 2,320
1
vote
0
answers
56
views
How to exact the data without duplication from mysql logs?
After I took a full backup of DB using mysqldump -u user -p database > backup.sql. After this I used
mysql> show master status;
+------------------+----------+--------------+------------------+...
- 103
2
votes
1
answer
2k
views
How can I perform multi-table delete in safe mode with binary logging enabled?
Let's pretend, that I have following data structure:
DROP TABLE IF EXISTS `A`;
DROP TABLE IF EXISTS `B`;
DROP TABLE IF EXISTS `C`;
CREATE TABLE IF NOT EXISTS `C` (
`ID_C`
INT UNSIGNED
...
- 12.2k
0
votes
1
answer
66
views
Need advice on binary logs?
We have set up master slave configuration and we are storing bin logs of 30 days. It is taking up more space on server and we need find some better way to handle it.
We are thinking to store logs of ...
- 1,361
5
votes
1
answer
2k
views
why issue 'reset master' when resyncing a mysql slave
Ive recently needed to perform some DB resync's and have a question regarding (what appears to be) the common practice of issuing a 'RESET MASTER' before dumping the DB on the master.
Just about all ...
- 195
4
votes
1
answer
2k
views
Gather ID of mysql table
Currently working on a binary log parser, that reads out the binary log written by a mysql database to write and convert changes into a MongoDB. This way I want to create "materialized views" ...
- 2,914
3
votes
1
answer
578
views
Is it possible to use the binlog from 1 schema and use mysqlbinlog to send them to another schema?
Each night we run mysqldump from production and dump it to another schema to use as a test or development server, on the same server. As the databases grow it is taking much of the night to finish. ...
- 31