can pull-replication work together with replication?

I have a multi-site gerrit and use replication ,but it`s too slow. i want use pull-replication, and i try it in test env. it work well.but pull-replication don`t
 save event file in disk like replication, so when some site restart. pull replication process don`t restart agant.i config replication and pull replication together. i don`t know this two plugin can work together? and is my configuration correct?

test node 1:
[remote "test-node-2"]
    url = root@test-node-2:/opt/gerrit_docker_site/git/${name}.git
    apiUrl = http://test-node-2
    fetch = +refs/*:refs/*
    mirror = true
    connectionTimeout = 60000
    idleTimeout = 10000
    timeout = 3600
    thread = 10
    rescheduleDelay = 15
    createMissingRepositories = true
    replicateProjectDeletions = true
    replicatePermissions = true

[remote "test-node-3"]
    url = root@test-node-3:/opt/gerrit_docker_site/git/${name}.git
    apiUrl = http://test-node-3
    fetch = +refs/*:refs/*
    mirror = true
    connectionTimeout = 60000
    idleTimeout = 10000
    timeout = 3600
    thread = 10
    rescheduleDelay = 15
    createMissingRepositories = true
    replicateProjectDeletions = true
    replicatePermissions = true

[remote "push-test-node"]
    url = root@test-node-2:/opt/gerrit_docker_site/git/${name}.git
    url = root@test-node-3:/opt/gerrit_docker_site/git/${name}.git
    push = +refs/*:refs/*
    mirror = true
    connectionTimeout = 60000
    idleTimeout = 10000
    timeout = 3600
    thread = 10
    replicationDelay = 60
    rescheduleDelay = 15
    createMissingRepositories = true
    replicateProjectDeletions = true
    replicatePermissions = true

[gerrit]
    autoReload = false
    replicateOnStartup = false
[replication]
    lockErrorMaxRetries = 5
    maxRetries = 10
    useCGitClient = false
    consumeStreamEvents = true
    #eventBrokerTopic = gerrit_pull
    excludeRefs = refs/multi-site/version
    syncRefs = refs/*
    eventsDirectory = /var/gerrit/replication_events


test node 2:
[remote "test-node-1"]
    url = root@test-node-1:/opt/gerrit_docker_site/git/${name}.git
    apiUrl = http://test-node-1
    fetch = +refs/*:refs/*
    mirror = true
    connectionTimeout = 60000
    idleTimeout = 10000
    timeout = 3600
    thread = 10
    rescheduleDelay = 15
    createMissingRepositories = true
    replicateProjectDeletions = true
    replicatePermissions = true

[remote "test-node-3"]
    url = root@test-node-3:/opt/gerrit_docker_site/git/${name}.git
    apiUrl = http://test-node-3
    fetch = +refs/*:refs/*
    mirror = true
    connectionTimeout = 60000
    idleTimeout = 10000
    timeout = 3600
    thread = 10
    rescheduleDelay = 15
    createMissingRepositories = true
    replicateProjectDeletions = true
    replicatePermissions = true

[remote "push-test-node"]
    url = root@test-node-1:/opt/gerrit_docker_site/git/${name}.git
    url = root@test-node-3:/opt/gerrit_docker_site/git/${name}.git
    push = +refs/*:refs/*
    mirror = true
    connectionTimeout = 60000
    idleTimeout = 10000
    timeout = 3600
    thread = 10
    replicationDelay = 60
    rescheduleDelay = 15
    createMissingRepositories = true
    replicateProjectDeletions = true
    replicatePermissions = true

[gerrit]
    autoReload = false
    replicateOnStartup = false
[replication]
    lockErrorMaxRetries = 5
    maxRetries = 10
    useCGitClient = false
    consumeStreamEvents = true
    #eventBrokerTopic = gerrit_pull
    excludeRefs = refs/multi-site/version
    syncRefs = refs/*
    eventsDirectory = /var/gerrit/replication_events


test node 3:
[remote "test-node-1"]
    url = root@test-node-1:/opt/gerrit_docker_site/git/${name}.git
    apiUrl = http://test-node-1
    fetch = +refs/*:refs/*
    mirror = true
    connectionTimeout = 60000
    idleTimeout = 10000
    timeout = 3600
    thread = 10
    rescheduleDelay = 15
    createMissingRepositories = true
    replicateProjectDeletions = true
    replicatePermissions = true

[remote "test-node-2"]
    url = root@test-node-2:/opt/gerrit_docker_site/git/${name}.git
    apiUrl = http://test-node-2
    fetch = +refs/*:refs/*
    mirror = true
    connectionTimeout = 60000
    idleTimeout = 10000
    timeout = 3600
    thread = 10
    rescheduleDelay = 15
    createMissingRepositories = true
    replicateProjectDeletions = true
    replicatePermissions = true

[remote "push-test-node"]
    url = root@test-node-1:/opt/gerrit_docker_site/git/${name}.git
    url = root@test-node-2:/opt/gerrit_docker_site/git/${name}.git
    push = +refs/*:refs/*
    mirror = true
    connectionTimeout = 60000
    idleTimeout = 10000
    timeout = 3600
    thread = 10
    replicationDelay = 60
    rescheduleDelay = 15
    createMissingRepositories = true
    replicateProjectDeletions = true
    replicatePermissions = true

[gerrit]
    autoReload = false
    replicateOnStartup = false
[replication]
    lockErrorMaxRetries = 5
    maxRetries = 10
    useCGitClient = false
    consumeStreamEvents = true
    #eventBrokerTopic = gerrit_pull
    excludeRefs = refs/multi-site/version
    syncRefs = refs/*
    eventsDirectory = /var/gerrit/replication_events

Hi Andy,
You mean the replication plugin latency is too high: there is no difference in Gerrit performance with the replication or pull-replication plugins.
You haven’t configured any broker-based catch-up mechanism, that is why the Gerrit restart does not consume missed replication events.

See an example of how it should be configured at [1]
They are fully independent, so there is no reason why they should not be able to work together.
However, if you have pull-replication configured, you don’t need the replication plugin anymore, unless you want to replicate to a non-Gerrit server.
No, you’re missing the 'eventBrokerTopic = <<TOPIC>>’ for allowing automatic catch-up upon restarts.

HTH

Luca.
[1] https://gerrit.googlesource.com/plugins/pull-replication/+/refs/heads/master/example-setup/broker/configs/replication.config.template

>
> --
> --
> To unsubscribe, email [email protected]
> More info at https://groups.google.com/group/repo-discuss?hl=en
>
> ---
> You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
> To view this discussion visit https://groups.google.com/d/msgid/repo-discuss/71c69a04-f6c1-4dcd-958e-63cbf77067fen%40googlegroups.com.

thank you, now i update config like this consumeStreamEvents = false and eventBrokerTopic = gerrit_replication , i still cat`t  see any kafka event in topic  gerrit_replication , pull replication still use reset api. 

Did you see the fully working example in the pull-replication folder I pointed at [1]?

The pull-replication *consumes* events, you need to also *put* those events to that topic, it doesn’t happen if you don’t tell Gerrit to do so.
See the example at [1] which contains all the configuration and components needed.

HTH

Luca.
> To view this discussion visit https://groups.google.com/d/msgid/repo-discuss/08ee996e-1609-4476-9114-5c467de81154n%40googlegroups.com.

thank. Finally, I achieved message sending by using consumeStreamEvents=true and was able to synchronize data even after other nodes were started. But I uploaded a large repository to node A, and after the upload was completed (nodes B and C have already started pull replication and have not yet completed it). I restarted all the nodes one by one, and the synchronization task for this large repository disappeared. He won't start again.

That’s not what the example [1] does and has drawbacks, like losing events if they are already consumed by not actioned.
I would recommend following the example [1].
How did you shutdown the nodes?
What version of the pull-replication plugin did you use?

The latest version has the control to hold the shutdown process until the replication events are complete.

HTH

Luca.

[1] https://gerrit.googlesource.com/plugins/pull-replication/+/refs/heads/master/example-setup/broker/configs/replication.config.template
> To view this discussion visit https://groups.google.com/d/msgid/repo-discuss/da8e4d7a-e2d4-49b6-a5f1-741132b8b39an%40googlegroups.com.

OK , thanks a lot, I will flow example [1] try again later ,I use pull replication version  stable-3.4, My gerrit version is 3.4.6,  I enter the  command docker restart  [container id] to simulate service downtime. you know when some exceptions occur, gerrit will not close normally, such as OOM.

You are right,  Maybe I can collect the pull-replication logs and write a scheduled monitoring tool to  check the data consistency.