1,861 questions
-1
votes
0
answers
12
views
Data Replication across data center for banking transactions
Suppose I am developing a backend for Transaction management system. When a banking transaction happen which when mapped to Db transaction, does it include write logs to different geos. If it does not ...
- 166
1
vote
0
answers
37
views
Suppress DBCC messages for sp_addarticle
Anyone know how to turn off this unwanted output? I have several stored procedures that are used to tear down and rebuild replication for hundreds of databases.
These stored procedures output lots of ...
- 35
0
votes
0
answers
25
views
Multiplexing read-only/read-write datasource
I've implemented CustomRoutingDataSource. It should multiplex between two DataSources depending on whether current transation is ReadOnly or not.
public class CustomRoutingDataSource extends ...
- 5,278
0
votes
1
answer
56
views
SymmetricDS How do I transform a MySQL Longtext column to a PostgreSQL Text column?
I am working on a replication between a old MySQL 8 database and a new PostgreSQL database.
This is a two way replication but I am having issues with syncing a LongText column in MySQL to a Text ...
- 1
0
votes
0
answers
20
views
Cannot delete or update a parent row: a foreign key constraint fails with Galera and MaxScale
I'm working with two servers MariaDB with galera replication and MaxScale rw router.
One server takes write queryies only, the other one just selects.
I have this sequence of operations:
start ...
- 9,474
0
votes
2
answers
42
views
SQL Server read only failover database CLR enabled
We have enabled CLR on the primary database with a read only synchronized readonly failover database and it works perfectly using the following instruction.
IF EXISTS (SELECT * FROM sys....
- 1,043
0
votes
0
answers
25
views
Handling Replication Lag with ProxySQL in a Spring Boot Application using MySQL InnoDB
I am working on a Spring Boot application using MySQL 8.0.39 with InnoDB and ProxySQL for load balancing between a master and a slave MySQL server. The database is configured to split read and write ...
- 1
1
vote
1
answer
59
views
walsender process and connections via bouncer in Postgres is not using TLS Encryption
I have been testing TLS encryption for my Postgres database. All the external client connections are using SSL for encryption. I tried to query the pg_stat_ssl view, which will list the information ...
- 11
0
votes
0
answers
37
views
SQL Server Replication procedure or function sp_get_redirected_publisher has too many arguments specified
Replication stopped working on a SQL Server 2022 to SQL Server 2022 setup. Snapshot replication, one subscriber. As far as I am aware nothing changed.
I am getting the following error message, every ...
0
votes
1
answer
44
views
Postgresql Replication
I have a table in production whose structure I want to modify. I can ALTER the table to add column but the column gets added at the end, may be the order may not matter. But at least visually I see it ...
- 1,011
0
votes
0
answers
35
views
Logical replication not working and no errors in the logs
I have setup a logical replication in PostgreSQL and have made the changes in postgres.conf where i have set wal_level = 'logical'
The create subscription part works well but the create subscriber ...
- 3,008
0
votes
0
answers
29
views
Postgres data sync between different servers
I have 2 postgres clusters (A and B) on different servers. Whenever data is written to the table on server A, I want to copy the data to the table on server B.
But the data retention in the table on ...
- 13
0
votes
1
answer
257
views
Postgres logical replication stuck at LSN value. I want to advance past it with pg_replication_origin_advance
Postgres 13 on AWS RDS
Replication slot is growing in size
CPU and I/O are low on both master and slave
The subscriber seems to be stuck at 33533/7D2841D8 even though the received LSN keeps advancing. ...
- 6,437
0
votes
0
answers
105
views
Timeline corruption PostgreSQL
I have this error, and I don't know how to fix it correctly. I saw people doing pg_basebackup everytime it happens. There is any better way?
2024-08-27 12:27:41.521 CEST [1] LOG: starting PostgreSQL ...
- 135
0
votes
1
answer
394
views
Postgres Hot standby stuck starting waiting for contrrecord despite wal file being present
We are experimenting with patroni Multi Datacenter HA with postgres16 using replication slots. At high level for building the remote cluster from scratch , we are by running base backup, while active ...
- 393
0
votes
1
answer
114
views
How to create notifications for Azure Data Sync errors or different way to schedule replication of Azure SQL data to on prem SQL Server?
We have a database in Azure SQL Server that we want to replicate continually to an On-Prem 2019 SQL Server. We are using Azure Data Sync but it fails every once in a while and we can't find out how to ...
0
votes
0
answers
45
views
Merkle Tree vs Replication Log / Event Log
I understand that Merkle Trees can be used to replicate data and guarantee
integrity of the data in byzantine environment (where you don't trust all the parties) and that systems like cansandra use it ...
- 41
0
votes
1
answer
166
views
Is Amazon Aurora RDS replication synchronous or asynchronous
We are using Amazon Aurora RDS with Postgres. We have Multi AZ deployment, with two instances, 1 writer and 1 reader replica. Now the question is, if the synchronization is synchronous or asynchronous ...
- 14.3k
0
votes
0
answers
111
views
DMS Source Data and Latency spike nightly
I'm using a SQL Server on-prem source for a DMS to replicate to a SQL Server RDS. Every night, at about midnight, I'm seeing a spike of ~1.6M CDC incoming changes on my DMS task, but nothing is making ...
0
votes
0
answers
16
views
In Quorum-Based Consensus Protocol, how does abort quorum get the updated data?
I am trying to understand "Quorum-Based Consensus Protocol". If there are total 5 nodes and 'Commit quorum' is of 3 nodes And 'Abort quorum' is also of 3 nodes. Now coordinator asks to ...
- 633
0
votes
2
answers
198
views
How to acquire exclusive lock in replicated mysql and clustered servers?
I have this scenario:
My php web application is deployed in a clustered environment, it uses a replicated mariadb database with galera.
It has to acquire an exclusive lock on database row resource and ...
- 9,474
0
votes
0
answers
36
views
Handling Stale Reads and Lost Updates in Galera Cluster During Large Transactions
I have a question about using Galera Cluster.
I understand that for large transactions (changing many rows), replication between nodes can take a long time. I also know that stale reads can occur in ...
0
votes
1
answer
258
views
MySQL master-slave replication with SSL: got error certificate verfication failed
Last_IO_Error: Error connecting to source 'repl@mysql-master:3306'. This was attempt 1/86400, with a delay of 60 seconds between attempts. Message: SSL connection error: error:0A000086:SSL routines::...
- 31
0
votes
0
answers
1k
views
pg_basebackup error: connection to server, no pg_hba.conf entry for replication connection
There are similar posts and answers to my issue, however they could not help me.
I am using PostgreSQL v16.3 and the latest version of Docker.
I am creating a very basic PostgreSQL cluster using a ...
- 88
0
votes
1
answer
326
views
DMS replication task fails due to a corrupt binlog event
I have a mariadb RDS database deployed as a multi-AZ in my AWS account, engine version 10.6.16. This DB serves as the main backend database serving ~100 connections per second. I also have a DMS ...
- 41
0
votes
0
answers
56
views
Error while adding subscriber in SQL Server replication process
I get this error while adding subscriber database in SQL Server:
Property Password cannot be changed or read after a connection string has been set. (Microsoft.SqlServer.ConnectionInfo)
Some ...
0
votes
0
answers
1k
views
replication slot's wal_status reserved vs extended
When queried using
SELECT * FROM pg_replication_slots ;
There's one more column as wal_status which has can be mainly either reserved vs extended.
what is the main difference?
will one has higher ...
- 229
1
vote
1
answer
46
views
How can I sync replicas with each other?
I have two replicas for my main cassandra server. Both the replicas are not in sync with the main server. This happened because the replica servers were running into errors cause by commit logs for 3 ...
- 11
0
votes
0
answers
20
views
Consistency in the case of 2 nodes going down simultaneously in 5 node cluster
Let's we have a 5 nodes cluster(one master + 4 slaves). Application is doing writes with min quoram on 3 nodes (master + 2 slave) to ensure consistency in the case of node failure.
For ease of ...
- 6,564
0
votes
0
answers
124
views
Cross cluster replication for Elastic search community edition
I am using elastic search community version. I want to implement cross cluster replication for disaster recovery. I gone through cross cluster replication of elastic search but this is supported only ...
- 5,731
0
votes
1
answer
28
views
Postgres subscriber node took some time to reflect publisher config change which leads to postgres restart issue
I am aware of the postgres config caveats mentioned here: The easiest way to ensure this does not become a problem is to have these parameters set on the standbys to values equal to or greater than on ...
- 1,621
1
vote
1
answer
107
views
MongoReplicaSet - emulate replication lag
I'm using mongodb-memory-server for the e2e-testing on the project and want to identify potential problems with replication lag. Is there any way we can emulate a large replication delay?
Maybe there ...
0
votes
1
answer
517
views
What happens with the old primary after a "pg_ctl promote" on a replica?
I have a PostgreSQL 11 streaming (ie. wal_type = replica) cascading replication scheme set up as follows:
db1 -> db2 -> db3
What happens to db1 and db3 if I execute a pg_ctl promote on db2?
...
- 2,483
0
votes
1
answer
79
views
does pg_restore + wal_type=replica create a consistent replica?
I have a PostgreSQL server that has a pg_dump scheduled every day at 3 AM (low traffic time window) for backup purposes. These dump files are gzipped and stored at a convenient location. This server ...
- 2,483
0
votes
1
answer
900
views
AWS DMS Task Migration Mapping Rules - Add-Column using metadata
I need to create a new column for each record that DMS processes, right now I have the following mapping rule:
{
"rule-type": "transformation",
"rule-id": &...
- 3,576
1
vote
0
answers
142
views
Changing binlog_format in MariaDB slave
I have mariadb 10.3 master and slave running in AWS EC2, whose binlog format is MIXED for both. I want to migrate this db to Aurora MySQL using DMS and understand DMS needs the format to be in ROW.
I ...
- 53
0
votes
0
answers
56
views
Kafka mirror maker: Data in some topics are not replicated
Using mirror maker to replicate data across two geo -redundant sites.
I am not talking about internal topics.
But, data in some of the external topics that are designed to be replicated are not ...
- 598
0
votes
0
answers
135
views
ProxySQL as a backend for Wordpress results in login out user from wordpress
The idea was to split read and writes to a master-master cluster that has backend load balancer (ProxySQL), also there is load balancer for frontend (HAProxy).
Frontend part work flawless.
Backend ...
- 49
1
vote
0
answers
57
views
How can I minimize data loss & data transfer when I failback with the requirement that I promote the old primary back to its primary status?
The definition of failback according to IBM docs is "the process of returning production to its original location after a disaster or a scheduled maintenance period".
From my reading of ...
- 653
-1
votes
2
answers
256
views
Postgresql Replication Slot grows in GB suddenly
We are using Qlik Replicate for replicating data from RDS Postgresql to Synapse.
There is logical replication slot involved in this process, so here is the case when we start the replication, the ...
0
votes
0
answers
33
views
How to replicate data from 1 VM into another VM on-prem with linux/ubunto server
VM = Application on Ruby on rails using docker, memory = m5 16xlarge, 256, 1tb
Both VM(Primary & Backup) will be deploy on on-prem.
I have requirement where i need to sync both VM's simultaneously ...
- 107
0
votes
0
answers
91
views
Replication postgresql database error connection between master to standby
Dear all I have found this issue, while I trying to do replication postgresql database. I have run this command pg_basebackup -h 192.168.47.155 -D /var/lib/postgresql/data -U replicator -P -v -R -X ...
0
votes
0
answers
61
views
Is there a specified order for data replication among different partitions in Cosmos DB?
I have 1 write region (e.g., US), and 2 read regions (e.g., US and EU). I have a collection, and I want to write into it, using different partition keys. I wonder, is there a specified order for data ...
- 3,707
0
votes
0
answers
286
views
SQL Server replication: Need to remove articles from the publication
I am replicating the data from the Database1 on Server1 to respectively, Database2 on Server2.
Now, the requirement is to remove certain tables form being replicated - so I will remove them from the ...
- 461
0
votes
0
answers
175
views
What does the value 30 indicate in the type column in the MSrepl_commands table?
Result Set of MSrepl_commands
I want to infer that whether its a insert/update/delete command . We need to check at day end the amount of data replicated per article by the transactional replication. ...
0
votes
0
answers
136
views
How to Achieve Master-Master Replication in MongoDB for Two Databases in Different Locations?
I'm working on a project where I need to deploy two MongoDB databases in different geographical locations. I want both databases to work independently, allowing reads and writes on both sides. However,...
- 1
0
votes
2
answers
325
views
Real time replication from on premises Oracle to AWS
Is there any service offering from AWS, that does real time replication of data from Oracle which is on premises to Oracle instance running on EC2 in AWS ? But the catch is, the table structures are ...
- 359
2
votes
0
answers
161
views
SymmetricDS - Detected losing row for batch [[batch]] for missing foreign key parent [[table]]: [[id]]
I'm having an issue with SymmetricDS when it comes to sync.
So the issue is as follows, I have an HQ (1) and one Agency (2). This issue does not happen in the HQ --> Agency route, and only in the ...
- 52
1
vote
0
answers
311
views
Problem replicating MySQL database to ClickHouse
I am trying to replicate a MySQL database to ClickHouse following these steps
I could do all configuration at MySQL and ClickHouse without any errors.
But when I try to test the integration, following ...
- 577
0
votes
1
answer
757
views
Switching Postgresql Logical Replication Slave to new Master without re-sync
Currently we have the following setup: A Postgresql DB "Master" and global distributed Postgresql Slaves, using Logical Replication for data replication. The master is read/write, the slaves ...
- 11