Giter VIP home page Giter VIP logo

Comments (16)

kshitij10496 avatar kshitij10496 commented on June 30, 2024 6

Wow! This thread reads like a blog post I always would want to refer for migrations to MongoDB Atlas. ❤️

from leprechaun.

icyflame avatar icyflame commented on June 30, 2024

All times are in JST (UTC+9)

  1. Create an account on https://www.mongodb.com/cloud/atlas/signup

  2. Open Heroku dashboard and open the mLab add-on page (this automatically logs you into the mLab account)

  3. Connect Atlas account to mLab

  4. 23:42: Put the target application in Maintenance mode

    image

  5. Start the Configuration migration wizard on Atlas

  6. 23:44: Complete the Configuration migration subsection of the wizard (Just tapping on the green button a few times without changing anything on the screen)

    image

    Scroll down and click the green “Confirm Target and Continue” button to create a free Atlas database (M0 tier). It will take about 7-9 minutes for the Atlas database to be created.

    Looks like this step will take a few minutes;

  7. 23:48: Configure the migration and run a "test run" (recommended)

    image

    image

from leprechaun.

icyflame avatar icyflame commented on June 30, 2024

Stuck on the Confirm Connectivity Step

$ docker run --rm --name mongo -it mongo:latest mongo "$MONGODB_URI"
MongoDB shell version v4.4.1
connecting to: mongodb://...<REMOVED>


*** It looks like this is a MongoDB Atlas cluster. Please ensure that your IP whitelist allows connections from your network.

Error: can't connect to new replica set master [REMOVED:27017], err: AuthenticationFailed: bad auth : Authentication failed. :
connect@src/mongo/shell/mongo.js:374:17
@(connect):2:6
exception: connect failed
exiting with code 1

from leprechaun.

icyflame avatar icyflame commented on June 30, 2024

I brought the back out of maintenance mode while I get through this Confirm Connectivity Step

from leprechaun.

icyflame avatar icyflame commented on June 30, 2024

Trying to connect using the same version as the MongoDB that's present on the cluster (4.2.10);

Found and fixed the issue! ✅

$ echo $MONGODB_URI
mongodb+srv://USERNAME:<password>@PATH/DATABASE?retryWrites=true&w=majority

$ echo $MONGODB_URI
mongodb+srv://USERNAME:PASSWORD@PATH/DATABASE?retryWrites=true&w=majority

In the connection string that I copied from Heroku, the password was not present. It had the literal string <password>. I generated a password on the Atlas website: Security > Database Access > Admin User > Edit > Generate Password

image

from leprechaun.

icyflame avatar icyflame commented on June 30, 2024

Confirmed connectivity ✅

$ docker run --rm --name mongo -it mongo:4.2.10 mongo "$MONGODB_URI"
MongoDB shell version v4.2.10
connecting to: mongodb://<CONNECT_STRING>?authSource=admin&compressors=disabled&gssapiServiceName=mongodb&replicaSet=atlas-gvc7sl-shard-0&retryWrites=true&ssl=true&w=majority
2020-11-07T15:20:18.792+0000 I  NETWORK  [js] Starting new replica set monitor for CLUSTER1,CLUSTER2,CLUSTER3
2020-11-07T15:20:18.792+0000 I  CONNPOOL [ReplicaSetMonitor-TaskExecutor] Connecting to CLUSTER1
2020-11-07T15:20:18.792+0000 I  CONNPOOL [ReplicaSetMonitor-TaskExecutor] Connecting to CLUSTER2
2020-11-07T15:20:18.792+0000 I  CONNPOOL [ReplicaSetMonitor-TaskExecutor] Connecting to CLUSTER3
2020-11-07T15:20:19.720+0000 I  NETWORK  [ReplicaSetMonitor-TaskExecutor] Confirmed replica set for CLUSTER1,CLUSTER2,CLUSTER3
Implicit session: session { "id" : UUID("a4330377-220e-4229-a1d1-07bda7863d71") }
MongoDB server version: 4.2.10
Welcome to the MongoDB shell.
For interactive help, type "help".
For more comprehensive documentation, see
        https://docs.mongodb.com/
Questions? Try the MongoDB Developer Community Forums
        https://community.mongodb.com
2020-11-07T15:20:21.479+0000 I  STORAGE  [main] In File::open(), ::open for '/home/mongodb/.mongorc.js' failed with No such file or directory
Error while trying to show server startup warnings: user is not allowed to do action [getLog] on [admin.]
MongoDB Enterprise atlas-gvc7sl-shard-0:PRIMARY> db.people.findOne();
{
        "_id" : ObjectId("5a4e28528fdf63769d0d3f64"),
        "roll" : "13ME33012",
        "email" : "[FILTERED]",
        "verifier" : "[FILTERED]",
        "emailtoken" : "[FILTERED]",
        "linksuffix" : "[FILTERED]",
        "step1complete" : true,
        "step1completedat" : ISODate("2018-01-04T13:14:14.038Z"),
        "step2complete" : true,
        "step2completedat" : ISODate("2018-01-04T13:12:50.541Z")
}
MongoDB Enterprise atlas-gvc7sl-shard-0:PRIMARY> db.people.count();
65
MongoDB Enterprise atlas-gvc7sl-shard-0:PRIMARY>
bye
2020-11-07T15:21:04.036+0000 E  -        [main] Error saving history file: FileOpenFailed: Unable to open() file /home/mongodb/.dbshell: No such file or directory

Looks like the database has already been migrated successfully to Atlas! ✅

from leprechaun.

icyflame avatar icyflame commented on June 30, 2024

Add the new connection string as the new env variable ATLAS_MONGODB_URI

$ heroku config:set ATLAS_MONGODB_URI=[FILTERED] -a leprechaun-metakgp
 ›   Warning: heroku update available from 7.39.0 to 7.47.0.
Setting ATLAS_MONGODB_URI and restarting ⬢ leprechaun-metakgp... done, v41
ATLAS_MONGODB_URI: [FILTERED]

$ heroku config -a leprechaun-metakgp
 ›   Warning: heroku update available from 7.39.0 to 7.47.0.
=== leprechaun-metakgp Config Vars
ATLAS_MONGODB_URI: [FILTERED]
AUTH_TOKEN:        [FILTERED]
BASE_LINK:         [FILTERED]
DB_NAME:           [FILTERED]
FROM_EMAIL:        [FILTERED]
FROM_NAME:         [FILTERED]
MONGODB_URI:       [FILTERED]
SENDGRID_API_KEY:  [FILTERED]

from leprechaun.

icyflame avatar icyflame commented on June 30, 2024

image

Ready for migration; I put the app in maintenance mode;

from leprechaun.

icyflame avatar icyflame commented on June 30, 2024

Migration completed; I changed the application code to look at only the new DB variable

4b5de19

from leprechaun.

icyflame avatar icyflame commented on June 30, 2024

Looks like application is having trouble connecting to this new MongoDB

2020-11-07T15:30:55.242705+00:00 app[api]: Release v42 created by user [email protected]
2020-11-07T15:30:55.424356+00:00 heroku[web.1]: Restarting
2020-11-07T15:30:55.444981+00:00 heroku[web.1]: State changed from up to starting
2020-11-07T15:30:56.144985+00:00 heroku[web.1]: Starting process with command `leprechaun`
2020-11-07T15:30:56.357398+00:00 heroku[web.1]: Stopping all processes with SIGTERM
2020-11-07T15:30:56.455798+00:00 heroku[web.1]: Process exited with status 143
2020-11-07T15:30:59.000493+00:00 app[web.1]: panic: unsupported connection URL option: retryWrites=true
2020-11-07T15:30:59.000551+00:00 app[web.1]:
2020-11-07T15:30:59.000645+00:00 app[web.1]: goroutine 1 [running]:
2020-11-07T15:30:59.000723+00:00 app[web.1]: main.DialDB(0x7d087e)
2020-11-07T15:30:59.000832+00:00 app[web.1]: /tmp/tmp.yuEgqp34LW/.go/src/github.com/icyflame/leprechaun/db.go:14 +0xad
2020-11-07T15:30:59.000884+00:00 app[web.1]: main.init.ializers()
2020-11-07T15:30:59.000996+00:00 app[web.1]: /tmp/tmp.yuEgqp34LW/.go/src/github.com/icyflame/leprechaun/main.go:10 +0x22
2020-11-07T15:30:59.050064+00:00 heroku[web.1]: Process exited with status 2
2020-11-07T15:30:59.098095+00:00 heroku[web.1]: State changed from starting to crashed
2020-11-07T15:30:59.102300+00:00 heroku[web.1]: State changed from crashed to starting
2020-11-07T15:30:59.619620+00:00 heroku[web.1]: Starting process with command `leprechaun`
2020-11-07T15:31:02.596028+00:00 app[web.1]: panic: unsupported connection URL option: w=majority
2020-11-07T15:31:02.596044+00:00 app[web.1]:
2020-11-07T15:31:02.596051+00:00 app[web.1]: goroutine 1 [running]:
2020-11-07T15:31:02.596052+00:00 app[web.1]: main.DialDB(0x7d087e)
2020-11-07T15:31:02.596052+00:00 app[web.1]: /tmp/tmp.yuEgqp34LW/.go/src/github.com/icyflame/leprechaun/db.go:14 +0xad
2020-11-07T15:31:02.596058+00:00 app[web.1]: main.init.ializers()
2020-11-07T15:31:02.596087+00:00 app[web.1]: /tmp/tmp.yuEgqp34LW/.go/src/github.com/icyflame/leprechaun/main.go:10 +0x22
2020-11-07T15:31:02.650252+00:00 heroku[web.1]: Process exited with status 2
2020-11-07T15:31:02.713442+00:00 heroku[web.1]: State changed from starting to crashed
2020-11-07T15:31:13.000000+00:00 app[api]: Build succeeded
2020-11-07T15:31:37.219119+00:00 app[api]: Set maintenance mode off by user [email protected]
2020-11-07T15:31:43.121092+00:00 heroku[router]: at=error code=H10 desc="App crashed" method=GET path="/" host=leprechaun.metakgp.org request_id=f520a829-f9f1-4b03-94fa-41ad6fc58e38 fwd="220.213.59.37,162.158.118.90" dyno= connect= service= status=503 bytes= protocol=http
2020-11-07T15:32:02.460228+00:00 heroku[router]: at=error code=H10 desc="App crashed" method=GET path="/" host=leprechaun.metakgp.org request_id=a291034c-80ed-4b99-88fb-a8231ad38420 fwd="220.213.59.37,162.158.118.90" dyno= connect= service= status=503 bytes= protocol=http

I will reset to mLab DB while I fix the issue

from leprechaun.

icyflame avatar icyflame commented on June 30, 2024

Driver needs to support at least MongoDB 3.6 with SSL

https://github.com/globalsign/mgo

https://docs.mlab.com/troubleshooting-atlas-connection-issues/#5-ensure-that-the-version-of-your-mongodb-driver-is-compatible-with-at-least-mongodb-36x

I can't find any other Go driver which supports Mongo DB 4.x

from leprechaun.

icyflame avatar icyflame commented on June 30, 2024

Alright, I was able to make it work by replacing mgo.v2 with it's forked version! ✅

99b27a8

from leprechaun.

icyflame avatar icyflame commented on June 30, 2024

mongodb+srv connection string does not work. (driver too old and probably doesn't support that)

I will use the connection string for older versions.

from leprechaun.

icyflame avatar icyflame commented on June 30, 2024

I didn't include the new package in the list of dependencies managed by govendor inside vendor/vendor.json

remote: Building source:
remote:
remote: -----> Go app detected
remote: -----> Fetching stdlib.sh.v8... done
remote: -----> Checking vendor/vendor.json file.
remote:  !!    The 'heroku.goVersion' field is not specified in 'vendor/vendor.json'.
remote:  !!
remote:  !!    Defaulting to go1.12.17
remote:  !!
remote:  !!    For more details see: https://devcenter.heroku.com/articles/go-apps-with-govendor#build-configuration
remote:  !!
remote: -----> Using go1.12.17
remote:  !!    Installing package '.' (default)
remote:  !!
remote:  !!    To install a different package spec set 'heroku.install' in 'vendor/vendor.json'
remote:  !!
remote:  !!    For more details see: https://devcenter.heroku.com/articles/go-apps-with-govendor#build-configuration
remote:  !!
remote: -----> Fetching any unsaved dependencies (govendor sync)
remote: -----> Running: go install -v -tags heroku .
remote: db.go:6:2: cannot find package "github.com/globalsign/mgo" in any of:
remote:         /tmp/tmp.0SAsRQWCYA/.go/src/github.com/icyflame/leprechaun/vendor/github.com/globalsign/mgo (vendor tree)
remote:         /tmp/codon/tmp/cache/go1.12.17/go/src/github.com/globalsign/mgo (from $GOROOT)
remote:         /tmp/tmp.0SAsRQWCYA/.go/src/github.com/globalsign/mgo (from $GOPATH)
remote: handlers.go:14:2: cannot find package "github.com/globalsign/mgo/bson" in any of:
remote:         /tmp/tmp.0SAsRQWCYA/.go/src/github.com/icyflame/leprechaun/vendor/github.com/globalsign/mgo/bson (vendor tree)
remote:         /tmp/codon/tmp/cache/go1.12.17/go/src/github.com/globalsign/mgo/bson (from $GOROOT)
remote:         /tmp/tmp.0SAsRQWCYA/.go/src/github.com/globalsign/mgo/bson (from $GOPATH)
remote:  !     Push rejected, failed to compile Go app.
remote:
remote:  !     Push failed
remote: Verifying deploy...
remote:
remote: !       Push rejected to leprechaun-metakgp.
remote:
To https://git.heroku.com/leprechaun-metakgp.git
 ! [remote rejected] prod -> master (pre-receive hook declined)
error: failed to push some refs to 'https://git.heroku.com/leprechaun-metakgp.git'
Switched to branch 'master'
Your branch is up to date with 'origin/master'.

from leprechaun.

icyflame avatar icyflame commented on June 30, 2024

Accidentally closed the issue;

Restarted Heroku dyno with connection string for the Atlas DB ✅

The app is up and working: https://leprechaun.metakgp.org/

from leprechaun.

icyflame avatar icyflame commented on June 30, 2024

image

I will leave the database on mLab until tomorrow morning and then delete the add-on from there too 🙆‍♂️

from leprechaun.

Related Issues (8)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.