alexandre-spieser / mongodb-generic-repository Goto Github PK

modify connectionString Value with code:
const string connectionString = "mongodb://username:pwd@host1:port1,host2:port2/db?replicaSet=mongoSet";

then run CoreIntegrationTests handler errors :

MongoDB.Driver.MongoConnectionException : An exception occurred while receiving a message from the server.
---- System.IO.EndOfStreamException : Attempted to read past the end of the stream.
at MongoDB.Driver.Core.Connections.BinaryConnection.ReceiveBuffer()
at MongoDB.Driver.Core.Connections.BinaryConnection.ReceiveBuffer(Int32 responseTo, CancellationToken cancellationToken)
at MongoDB.Driver.Core.Connections.BinaryConnection.ReceiveMessage(Int32 responseTo, IMessageEncoderSelector encoderSelector, MessageEncoderSettings messageEncoderSettings, CancellationToken cancellationToken)
at MongoDB.Driver.Core.ConnectionPools.ExclusiveConnectionPool.AcquiredConnection.ReceiveMessage(Int32 responseTo, IMessageEncoderSelector encoderSelector, MessageEncoderSettings messageEncoderSettings, CancellationToken cancellationToken)
at MongoDB.Driver.Core.WireProtocol.CommandUsingQueryMessageWireProtocol1.Execute(IConnection connection, CancellationToken cancellationToken) at MongoDB.Driver.Core.WireProtocol.CommandWireProtocol1.Execute(IConnection connection, CancellationToken cancellationToken)
at MongoDB.Driver.Core.Servers.Server.ServerChannel.ExecuteProtocol[TResult](IWireProtocol1 protocol, CancellationToken cancellationToken) at MongoDB.Driver.Core.Servers.Server.ServerChannel.Command[TResult](ICoreSession session, ReadPreference readPreference, DatabaseNamespace databaseNamespace, BsonDocument command, IEnumerable1 commandPayloads, IElementNameValidator commandValidator, BsonDocument additionalOptions, Action1 postWriteAction, CommandResponseHandling responseHandling, IBsonSerializer1 resultSerializer, MessageEncoderSettings messageEncoderSettings, CancellationToken cancellationToken)
at MongoDB.Driver.Core.Operations.RetryableWriteCommandOperationBase.ExecuteAttempt(RetryableWriteContext context, Int32 attempt, Nullable1 transactionNumber, CancellationToken cancellationToken) at MongoDB.Driver.Core.Operations.RetryableWriteOperationExecutor.Execute[TResult](IRetryableWriteOperation1 operation, RetryableWriteContext context, CancellationToken cancellationToken)
at MongoDB.Driver.Core.Operations.BulkUnmixedWriteOperationBase1.ExecuteBatches(RetryableWriteContext context, CancellationToken cancellationToken) at MongoDB.Driver.Core.Operations.BulkUnmixedWriteOperationBase1.Execute(RetryableWriteContext context, CancellationToken cancellationToken)
at MongoDB.Driver.Core.Operations.BulkMixedWriteOperation.ExecuteBatch(RetryableWriteContext context, Batch batch, CancellationToken cancellationToken)
at MongoDB.Driver.Core.Operations.BulkMixedWriteOperation.Execute(IWriteBinding binding, CancellationToken cancellationToken)
at MongoDB.Driver.OperationExecutor.ExecuteWriteOperation[TResult](IWriteBinding binding, IWriteOperation1 operation, CancellationToken cancellationToken) at MongoDB.Driver.MongoCollectionImpl1.ExecuteWriteOperation[TResult](IClientSessionHandle session, IWriteOperation1 operation, CancellationToken cancellationToken) at MongoDB.Driver.MongoCollectionImpl1.BulkWrite(IClientSessionHandle session, IEnumerable1 requests, BulkWriteOptions options, CancellationToken cancellationToken) at MongoDB.Driver.MongoCollectionImpl1.<>c__DisplayClass23_0.b__0(IClientSessionHandle session)
at MongoDB.Driver.MongoCollectionImpl1.UsingImplicitSession[TResult](Func2 func, CancellationToken cancellationToken)
at MongoDB.Driver.MongoCollectionImpl1.BulkWrite(IEnumerable1 requests, BulkWriteOptions options, CancellationToken cancellationToken)
at MongoDB.Driver.MongoCollectionBase1.<>c__DisplayClass64_0.<InsertOne>b__0(IEnumerable1 requests, BulkWriteOptions bulkWriteOptions)
at MongoDB.Driver.MongoCollectionBase1.InsertOne(TDocument document, InsertOneOptions options, Action2 bulkWrite)
at MongoDB.Driver.MongoCollectionBase1.InsertOne(TDocument document, InsertOneOptions options, CancellationToken cancellationToken) at MongoDbGenericRepository.DataAccess.Create.MongoDbCreator.AddOne[TDocument,TKey](TDocument document) in D:\development\mongodb-generic-repository\MongoDbGenericRepository\DataAccess\Create\MongoDbCreator.cs:line 49 at CoreIntegrationTests.Infrastructure.MongoDbTKeyDocumentTestBase2.AddOne() in D:\Public\mongodb-generic-repository\CoreIntegrationTests\Infrastructure\MongoDbTKeyDocumentTestBase.cs:line 78
----- Inner Stack Trace -----
at MongoDB.Driver.Core.Misc.StreamExtensionMethods.ReadBytes(Stream stream, Byte[] buffer, Int32 offset, Int32 count, CancellationToken cancellationToken)
at MongoDB.Driver.Core.Connections.BinaryConnection.ReceiveBuffer()

We just hit a pretty big production issue today would love to request a change to your library.

Essentially, we have data access code that is not utilizing this packages repository classes ( and i'm sure other consumers would as well especially if they are following a strict form of CQRS ). Unfortunately, The default settings for the mongodb C# client is to use LegacyUUID. Unfortunately, as soon as an implementation of our repository gets instantiated the below lines of code is GLOBALLY changing the driver settings mid application run.

Is this a requirement for your library to work correctly? This is just a bad practice in general as it causes mongodb C# driver ( hence the application ) to start behaving differently mid application run. In our case, we were creating user sessions using the legacy uuid default and when our repository got instantiated the updating/reading of sessions created with LegacyUUID started failing preventing customers from checking out.

I am trying to MOQ the BaseMongoRepository methods to have a unit test for my application, but I cannot MOQ the methods, is there an example available for creating MOQ of the methods, in the repository? Thank you

Hello,
first of all congrats for your great work.

is it possible to do something like
select Name, Color from Cars;

instead of
select * from Cars;

thanks.

How can i use this function

                bool result = await _mongoDbRepository.UpdateOneAsync<XPartitionedDocument,float>(
                    documentToModify: isExist,
                    field: x => (float)x.TotalPrice,
                    value: td.TotalPrice
                    );

https://stackoverflow.com/questions/54885343/mongodb-updateoneasync-expressionfunction-returning-input-string-was-not-in-a-co

Hi, any chances that there will be security updates?

Currently, the used MongoDB.Driver version is affected by vulnerabilities. Same applies for SharpCompress.

Looking forward to hear from you Alexandre.

Hey,

Is Upsert possible on the repository currently? For Example, if i want to allow a single method to be called for both inserts AND updates? I tried using .UpdateOne() but that seems to not do this.

Is there a way to query the db with your repository in order to get the item with max field value.
For exemple:
you have a colection of stuff: { username, stuffName, orderColumn}
I would like to get the max order number for userName X. I've saw that the driver would support a findOne sortby : {orderColumn :-1} but I cannot figure a way to do that with this repository.

Can you please provide and example? Can you also document this for future users?

Hi, is it possible to implement Discriminator?

Thank you

[BsonDiscriminator(RootClass = true)]
[BsonKnownTypes(typeof(MyClass1), typeof(MyClass2))]
public class MyClass
{
}

why didn't you consider context as an interface?
because i think if you consider it as an interface
it will decoupling and decrease dependencies and you can do unit testing and mocking

im rookie and plz clear me if i am wrong

Hi, thank you for the nice library, I have a question though.
Is it possible to save documents of the same type into different tables / collections depending on some property?

Example

class Fruit : Document 
{
  string Name { get; set; }
}

var pears = new List<Fruit>(); 
var apples = new List<Fruit>(); 

pears.Add(new Fruit { Name = "Pear 1" }); 
pears.Add(new Fruit { Name = "Pear 2" }); 

apples.Add(new Fruit { Name = "Apple 1" });
apples.Add(new Fruit { Name = "Apple 2" }); 

repository.AddManyAsync<Fruit>(pears);
repository.AddManyAsync<Fruit>(apples);

By default both, apples and pears, will be saved to collection Fruits.
Is it possible to save apples as Apples and pears as Pears?

I see that it's possible to add some prefix to a collection if I add documents one by one.
https://github.com/alexandre-spieser/mongodb-generic-repository#partitioned-collections

var apple = new Apple("Apple");
repository.AddOne(apple); 

Is it possible to do this for more than one document, e.g. for the List?

Hi! What performance reprecussions should I expect when the repository is configured to be scoped instead of singleton?

In my scenario, I'm trying to implement multitenancy (DB per tenant strategy) with AspNetCore.Identity.MongoDbCore, and I'd like to use a tenant-specific, scoped IMongoDbContext. To make it work I had to change both the IMongoDbContext and IMongoRepository services scoped.

My solution works, and I don't see any I/O bound operations in the Repository initialization, so I'm wondering why you advise it being a singleton. I'm new to MongoDB, so I may be missing something. Thank you in advance!

Hey Alexandre ,
First of all i would like to thanks you for excellent implementation of MongoDb repository (one of the best i have seen so far).

I would like you to suggest to split your current implemented BaseMongoRepository class to
BaseRepository<TKey>, ReadOnlyMongoRepository<TKey> and make a GuidBaseRepository:BaseRepository<Guid> as a example of implementation,

Another thing please expose FormatDocument<TDocument> as virtual in order to let repository use more generic.

Since there is many existing projects (including our's) that currently have stored data with keys that is not Guid but (ObjectId,Int, string etc...) that would like to move to your Generic repository implementation.
Current implementation exposes Guid (TKey) as Default key option.
By hard enclosing <TDocument,TKey> with Guid extensions in BaseRepository you are making harder to use it, since it is not possible to override your default <TDocument,TKey> extensions in BaseRepository developer can by mistake use Guid TKey instead of desired.

And in order to overcome current behavior flow ( and expose it to simple developers) we had to virtually rewrite BaseRepository and ReadOnlyMongoRepository since exposure of key type on each interaction with repository is not desired.

Thank's,
Shkard Evgeny

Hi Alexandre,

How to create MongoDB stored procedure or calling a Stored Procedure in MongoDB via "mongodb-generic-repository" ?

I'm stuck with adding new documents to the collection. When I use AddOneAsync(myDocument) it doesn't use my Id but stores a new Guid. I really could need some help.

I use:

• MongoDb.Driver V 2.10.3
• MongoDbGenericRepository V 1.4.3
• Newton.Json.Bson V 1.0.2
• Newton.Json V 12.0.3
• NetStandard 2.1

Here are my model and my model-base class:

    [CollectionName("Ride")]
    internal class RideModel : ModelBase
    {
        public Guid TripId { get; set; }
        public Guid? NextTripId { get; set; }
        public Guid VehicleId { get; set; }
        ...
    }

    internal abstract class ModelBase : IDocument
    {
        [BsonId]
        public Guid Id { get; set; }
        
        public int Version { get; set; }
    }

I made my own base repository inherited from BaseMongoRepository like this:

    internal abstract class RepositoryBase<TEntity, TModel> : BaseMongoRepository<Guid>, IRepository<TEntity>
        where TEntity: IEntity
        where TModel: ModelBase
    {
        private readonly ILogger logger;

        protected RepositoryBase(
            ILogger logger,
            IMongoDatabase mongoDatabase) : base(mongoDatabase)
        {
            this.logger = logger;
        }

        public virtual async Task SaveEntityAsync(TEntity entity)
        {
            var model = await GetByIdAsync<TModel>(entity.Id);

            var updatedModel = MapTo(entity);
            if (model != null)
            {
                await UpdateOneAsync(updatedModel);
                logger.LogInformation("{entityName} [{id}] updated.", typeof(TEntity).Name, updatedModel.Id, updatedModel);
            }
            else
            {
                await AddOneAsync(updatedModel);
                logger.LogInformation("{entityName} [{id}] added.", typeof(TEntity).Name, updatedModel.Id, updatedModel);
            }
        }

        [return: MaybeNull]
        public virtual async Task<TEntity> GetByIdAsync(Guid id)
        {
            var model = await GetByIdAsync<TModel>(id);
            return model == null ? default : MapTo(model);
        }

        public virtual async Task<IReadOnlyCollection<TEntity>> GetAllAsync()
            => (await base.GetAllAsync<TModel>(e => true))
                .Where(e => e != default)
                .Select(MapTo)
                .ToList();

        public virtual async Task<bool> UpdateAsync(TEntity entity) => 
            await base.UpdateOneAsync(MapTo(entity));

        public virtual async Task InsertAsync(TEntity entity) => 
            await base.AddOneAsync(MapTo(entity));

        public async Task DeleteAsync(TEntity entity)
            => await base.DeleteOneAsync(MapTo(entity));

        protected abstract TEntity MapTo(TModel model);
        
        protected abstract TModel MapTo(TEntity entity);
    }

I checked multiple times that the Id property is set to my Gui value (so MapTo() works). When I now call InsertAsync() or SaveEntityAsync() I always get a new guid in _id instead of the Guid value I set in the Id property. I guess that means MongoDb does not get that I have BsonId or unique Id and generate it own.

But why? I don't see what's wrong here. Can you please help?
Cheers, Marc

Hi,

You intend to implement a transaction-like pattern such as two-phase commit?

I try to use Aggegate with Database.GetCollection(GetCollectionName(partitionKey)) but it get this error:
Cannot implicitly convert type 'MongoDB.Driver.IAggregateFluent' to 'MongoDB.Driver.IMongoCollection'
How can I use AllowDiskUse?

I can't get the docs to show up in CosmosDB. I'm getting this error: "Error querying documents: The GuidRepresentation for the reader is CSharpLegacy, which requires the binary sub type to be UuidLegacy, not UuidStandard"

I tried below code, but it's not working:

public class MongoDbRepository : BaseMongoRepository, IDocumentDbRepository
{
public MongoDbRepository(string connectionString, string databaseName) : base(connectionString, databaseName)
{
MongoDbContext.SetGuidRepresentation(MongoDB.Bson.GuidRepresentation.CSharpLegacy);
}
}

Hi @alexandre-spieser

I manage customer stock and would like to know how I get the sum of the quantities in the collection. Here is a more or less example:

My Class

public class StockMovement : Document
    {
        public StockMovement()
        {
            Version = 1;
        }

        public Tenant Tenant { get; set; }
        public Company Company { get; set; }
        public Stock Stock { get; set; }
        public Product Product { get; set; }

        /// <summary>
        /// Entry, // Purchase
        /// Departure // Orders
        /// </summary>
        [BsonRequired]
        public Type MovementType { get; set; }

        /// <summary>
        /// My Quatity is NEGATIVE for MovementType "Departure" Sales/Orders for customers
        /// </summary>
        [BsonRepresentation(BsonType.Decimal128)]
        public decimal Quantity { get; set; }

        /// <summary>
        /// My UnitaryValue is NEGATIVE for MovementType "Departure" Sales/Orders for customers
        /// </summary>
        [BsonRepresentation(BsonType.Decimal128)]
        public decimal UnitaryValue { get; set; }

        public string Observation { get; set; }

        public User AddedBy { get; set; }
        public DateTime UpdatedAtUtc { get; set; }
        public User UpdatedBy { get; set; }
        public DateTime DeletedAtUtc { get; set; }
        public User DeletedBy { get; set; }
    }

My Service

public decimal GetQuantityByProductInStock(Guid productId, Guid companyId, Guid tenantId)
        {
            //Useless
            //var currentProduct = _repo.GetById<Product>(produtoId);

            var quantityInStock = _repo.GetAll<StockMovement>(w =>
                    w.Tenant.Id == tenantId && w.Company.Id == companyId && w.Product.Id == productId)
                .Sum(s => s.Quantity);

            if (quantityInStock <= 0)
                throw new Exception($@"NO stock :(");

            return quantityInStock;
        }

To get the result is done the sum of all the moves but, the customer already has more than 20k entries. How can I improve this?

I considered using asynchronous but, I do not know very well. Is it the only way out?

Can you help me please?

Hi,

Is it possible to use with .Net 4.8? I'm having some issues trying to run it mainly with libzstd.dll because when I installed using nuget it's putting as a content in my solution.

Thanks.

Hi,

I'm currently facing issue with UUID 04 representation in CosmosDB mongo API driver:

"Message": "GuidRepresentation Standard is only valid with subType UuidStandard, not with subType UuidLegacy.\r\nИмя параметра: guidRepresentation"

They said they added it to all regions: https://feedback.azure.com/forums/263030-azure-cosmos-db/suggestions/20206180-support-uuid-subtype-4

I just wonder how to make it work both MongoDB and CosmosDB with Standard UUID type?

Seems to me that only CSharpLegacy type will work for both. Any suggestions?

Hei Alexandre,
Current implementation of BaseMongoRepository does not allow to override MongoDb accessor properties (CUD)
can you kindly add setter to:
protected MongoDbUpdater MongoDbUpdater { get; }
protected MongoDbEraser MongoDbEraser { get; }
protected MongoDbCreator MongoDbCreator { get; }
protected MongoDbIndexHandler MongoDbIndexHandler { get; }

or allow inject extended functionality from constructor.

The main purpose of extension is to add transaction (MongoDb 4) functionality without many re-wrights in to our existing repositories.
ExtendedDataAccess.zip

Hi @alexandre-spieser,

How do I report the security protocol in connection with the mongodb atlas? Like this:

Your implementation at MongoDbContext, line 47:

Thanks. o/

I see in MongoDB there is asynchronous support. However when you build this Repo Generic I did not find the CRUD methods using the CancellationToken class. If my understanding is limited, please ignore it

When I call any generic methods on a repository does not show any docs

public interface ITestRepo : IBaseMongoRepository
{
        
}
public class Test : BaseMongoRepository,ITestRepo
{
        public Test(string connectionString, string databaseName = null) : base(connectionString, databaseName)
        {  }
}

// when calling using ITestRepo instance ( lets say _testRepo)

var result= _testRepo.GetAll<Document>(f => true);  // hovering on GetAll method does not show any docs, but if I use 1.4.3 version which works as expected

I would be nice to be able to utilize UpdateMany() on the base class just like AddMany()

Hello,

The nuget package you are building and publishing on nuget.org contains a bunch of thirdparty dependencies inside of it. The right way is to create a nuget package with your assemblies in it ONLY and utilizing the nuspec file to specify what specific versions of third party dependencies your package works with and letting nuget.exe CLI do the magic for people consuming this package.

Does this make sense?

Will the generic repository in its current guise support GridFS? If it does, could you possibly provide a sample of code time permitting.

I am looking to store binary data which in the main I will restrict to files of less than 10MB which can be stored in a standard document without issue. However, I am looking at users submitting multiple files which as part of the 'submit' process I create an archive of all these files - stamp them and store as a single zip file. In doing this I can potential exceed the maximum document size allowed.

GridFS would support this scenario. To be honest, I could change this mechanism to say zip on the fly when the request for a specific user submission containing multiple files is made or potentially create a home grown mechanism utilising 'chunking' however based on my limited understanding of Mongo this is what GridFS does under the hood anyways.

Thoughts?

Thanks.

I have collection which contain array of another model, and want to update all object in that array or delete them by ID

I want to update all category which id is 1, with this code I can update only first record in array ,
I used [-1] is ,it possible to update more than one item array

return await _memoCodeSetRepository.GetAndUpdateOne<MemoCodeSet>( Builders<MemoCodeSet>.Filter.Where(x => x.Id == modified.Id && x.MemoCode.Any(i => i.Category.Id == categoryGuid)), Builders<MemoCodeSet>.Update.Set(x => x.MemoCode[-1].Category, memoCodeSetCategory), new FindOneAndUpdateOptions<MemoCodeSet, MemoCodeSet> { ReturnDocument = ReturnDocument.After });

Example

{ "description": "Memo Code Set Test 2", "category": [ { "id": "1", "name": "Category Test 1" }, { "id": "1", "name": "Category Test 1" }, { "id": "fb843a50-4b91-44b9-ad9a-6422d4587983", "name": "test" }, { "id": "ba845800-4aa7-43fa-8e1b-f5759a6bf4de", "name": "test 2" }, { "id": "f4750605-38ba-4cd3-8cb4-8f7ea9fd6601", "name": "dsadsadasdsad" } ], "memoCode": [ { "id": "893f2a78-abf9-46a5-a15e-c613f286939c", "category": { "id": "e3230886-9df9-473a-b4b9-d933d1dda3a2", "name": "Category Test 1" }, "code": "6812", "description": "Description 1", "overTimeAbsenceCode": "Overtime", "exportCode": "23115", "categoryName": "Category Test 1" }, { "id": "18055ffb-69f6-4aea-8934-3b16d396b462", "category": { "id": "d19270ec-41f9-4afe-bf3a-4542512a8dc0", "name": "Category Test 2" }, "code": "2341", "description": "Description 2", "overTimeAbsenceCode": "Absence", "exportCode": "6513", "categoryName": "Category Test 2" } ], "copyMemoCodeSetId": null, "id": "e5aca50a-5032-4fd3-ac4f-5ad328fadbf7", "addedAtUtc": "2019-03-14T15:06:31.281Z", "version": 0 }

It seems that GetPaginated allows only for filtering but not sorting which kinda makes it unusable for typical grid case :/

When you have nested documents, should both classes implement IDocument? Do you need to set the BsonId for the ID property of the nested object?