Giter VIP home page Giter VIP logo

techdivision / import-category Goto Github PK

View Code? Open in Web Editor NEW
6.0 13.0 4.0 479 KB

This library provides the basic function for the Magento 2 category import. Besides the possibility to create root categories, the translations and URL rewrites can be imported. Similar to the product import, the Library also supports the upload of category images.

Home Page: https://m2if.com

License: MIT License

PHP 100.00%
magento import categories csv pacemaker

import-category's Introduction

Pacemaker - Category Import

Latest Stable Version Total Downloads License Build Status Scrutinizer Code Quality Code Coverage

Please visit the Pacemaker website or our documentation for additional information

Multistore URL Rewrite Import

  • Importer supports multistore url_key, just note that if the file contains only default store row for a new category, only one entry for defualt store is stored in the catalog_product_entity_varchar table and a key is stored in the url_rewrite table for all stores.
  • When updating this category with a new key for this category, it will be changed for all stores.
  • To avoid this problem, the default line can be omitted and only the specific store imported.

Special case

In some cases the default row must be included in the product file, in which case, if there are no entries in the varchar for the stores already, the url_rewrite for all stores is updated based on what is in the default column.

Possible solution

  • To solve the problem only the default store and the specific store should be imported and so the category will only be imported for the store and the specific store.
  • import should always include all stores in a new category, so each store has an entry in the catalog_product_entity_varchar table for the url_key attribute.
  • If the category is only to be updated in a store, the default row must be supplied with the specific store, then the url_rewrtie and url_key are updated in the Varchar table.

import-category's People

Contributors

adhocore avatar kenza-ya avatar mardl avatar pathmissing avatar wagnert avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

import-category's Issues

Invalid access to `is_autogenerated` and `redirect_type` fields

[03-Dec-2017 15:55:50 Europe/Berlin] [2017-12-03 15:55:50] logger/system.INFO: Now start import with serial dad0f052-d559-40dd-915f-849d3958e1e5 [catalog_category => add-update] [] []
[03-Dec-2017 15:55:50 Europe/Berlin] [2017-12-03 15:55:50] logger/system.INFO: Now start processing file projects/sample-data/tmp/dad0f052-d559-40dd-915f-849d3958e1e5/category-import_20161024-194026_01.csv [] []
[03-Dec-2017 15:55:50 Europe/Berlin] [2017-12-03 15:55:50] logger/system.INFO: Successfully processed file projects/sample-data/tmp/dad0f052-d559-40dd-915f-849d3958e1e5/category-import_20161024-194026_01.csv with 39 lines in 0,072560 s [] []
[03-Dec-2017 15:55:50 Europe/Berlin] [2017-12-03 15:55:50] logger/system.INFO: Now start processing file projects/sample-data/tmp/dad0f052-d559-40dd-915f-849d3958e1e5/category-path_20161024-194026_01.csv [] []
[03-Dec-2017 15:55:50 Europe/Berlin] [2017-12-03 15:55:50] logger/system.INFO: Successfully processed file projects/sample-data/tmp/dad0f052-d559-40dd-915f-849d3958e1e5/category-path_20161024-194026_01.csv with 39 lines in 0,011088 s [] []
[03-Dec-2017 15:55:50 Europe/Berlin] [2017-12-03 15:55:50] logger/system.INFO: Now start processing file projects/sample-data/tmp/dad0f052-d559-40dd-915f-849d3958e1e5/url-rewrite_20161024-194026_01.csv [] []
[03-Dec-2017 15:55:50 Europe/Berlin] PHP Warning:  Illegal string offset 'is_autogenerated' in /Users/wagnert/Workspace/import-cli-simple/vendor/techdivision/import-category/src/Observers/UrlRewriteUpdateObserver.php on line 95
[03-Dec-2017 15:55:50 Europe/Berlin] PHP Warning:  Illegal string offset 'redirect_type' in /Users/wagnert/Workspace/import-cli-simple/vendor/techdivision/import-category/src/Observers/UrlRewriteUpdateObserver.php on line 104
[03-Dec-2017 15:55:50 Europe/Berlin] PHP Warning:  Illegal string offset 'is_autogenerated' in /Users/wagnert/Workspace/import-cli-simple/vendor/techdivision/import-category/src/Observers/UrlRewriteUpdateObserver.php on line 95
[03-Dec-2017 15:55:50 Europe/Berlin] PHP Warning:  Illegal string offset 'is_autogenerated' in /Users/wagnert/Workspace/import-cli-simple/vendor/techdivision/import-category/src/Observers/UrlRewriteUpdateObserver.php on line 95
[03-Dec-2017 15:55:50 Europe/Berlin] PHP Warning:  Illegal string offset 'redirect_type' in /Users/wagnert/Workspace/import-cli-simple/vendor/techdivision/import-category/src/Observers/UrlRewriteUpdateObserver.php on line 104
[03-Dec-2017 15:55:50 Europe/Berlin] PHP Warning:  Illegal string offset 'is_autogenerated' in /Users/wagnert/Workspace/import-cli-simple/vendor/techdivision/import-category/src/Observers/UrlRewriteUpdateObserver.php on line 95
[03-Dec-2017 15:55:50 Europe/Berlin] PHP Warning:  Illegal string offset 'is_autogenerated' in /Users/wagnert/Workspace/import-cli-simple/vendor/techdivision/import-category/src/Observers/UrlRewriteUpdateObserver.php on line 95
[03-Dec-2017 15:55:50 Europe/Berlin] PHP Warning:  Illegal string offset 'is_autogenerated' in /Users/wagnert/Workspace/import-cli-simple/vendor/techdivision/import-category/src/Observers/UrlRewriteUpdateObserver.php on line 95
[03-Dec-2017 15:55:50 Europe/Berlin] PHP Warning:  Illegal string offset 'is_autogenerated' in /Users/wagnert/Workspace/import-cli-simple/vendor/techdivision/import-category/src/Observers/UrlRewriteUpdateObserver.php on line 95
[03-Dec-2017 15:55:50 Europe/Berlin] PHP Warning:  Illegal string offset 'redirect_type' in /Users/wagnert/Workspace/import-cli-simple/vendor/techdivision/import-category/src/Observers/UrlRewriteUpdateObserver.php on line 104
[03-Dec-2017 15:55:50 Europe/Berlin] PHP Warning:  Illegal string offset 'is_autogenerated' in /Users/wagnert/Workspace/import-cli-simple/vendor/techdivision/import-category/src/Observers/UrlRewriteUpdateObserver.php on line 95
[03-Dec-2017 15:55:50 Europe/Berlin] PHP Warning:  Illegal string offset 'redirect_type' in /Users/wagnert/Workspace/import-cli-simple/vendor/techdivision/import-category/src/Observers/UrlRewriteUpdateObserver.php on line 104
[03-Dec-2017 15:55:50 Europe/Berlin] [2017-12-03 15:55:50] logger/system.INFO: Successfully processed file projects/sample-data/tmp/dad0f052-d559-40dd-915f-849d3958e1e5/url-rewrite_20161024-194026_01.csv with 39 lines in 0,015016 s [] []
[03-Dec-2017 15:55:50 Europe/Berlin] [2017-12-03 15:55:50] logger/system.INFO: Found 4 files to archive in directory projects/sample-data/tmp/dad0f052-d559-40dd-915f-849d3958e1e5 [] []
[03-Dec-2017 15:55:50 Europe/Berlin] [2017-12-03 15:55:50] logger/system.INFO: Successfully created archived archive directory projects/sample-data/tmp/archive [] []
[03-Dec-2017 15:55:50 Europe/Berlin] [2017-12-03 15:55:50] logger/system.INFO: Successfully archived imported files to projects/sample-data/tmp/archive/dad0f052-d559-40dd-915f-849d3958e1e5.zip! [] []
[03-Dec-2017 15:55:50 Europe/Berlin] [2017-12-03 15:55:50] logger/system.INFO: Successfully finished import with serial dad0f052-d559-40dd-915f-849d3958e1e5 in 0,272176 s [] []

Position is set to 0

When importing the attached file, the leads to catalog_category_entity.position being zero. It happens for all categories. So if multiple categories are imported, they all don't have position information.

As a side effect it is no longer possible to reorder those categories in the admin panel via drag'n'drop.

Most recent import version; Magento version 2.3.4-p2 CE

category-import_20200714_1.zip

image

import creates URL rewrites for store views that don't contain the category

Prerequisites:

  • Magento 2.3
  • PHP 7.2
  • composer require techdivision/import-cli-simple:3.1.0-beta9
  • at least 2 websites (within the same Magento installation), say website A and website B
  • a number of store views a_1, ..., a_m in website A and b_1, ..., b_n in website B
  • a category X in website A and a category Y in website B, such that X and Y have the same URL keys

Expected behaviour:

When importing the categories A and B via M2IF the import should create URL rewrites for category A in the store views a_1, ..., a_m. And it should create URL rewrites for category B in the store views b_1, ..., b_n. That way there is no conflict with A and B having the same URL key.

Actual behaviour:

When importing the categories A (first) and B (second), the import first creates URL rewrites for A in all store views (a_1, ..., a_m, b_1, ..., b_n). Then it tries to create the rewrites for the category B and a integrity constraint exception is thrown.

Code details:

The problem lies in the class TechDivision\Import\Category\Observers\CategoryUrlRewriteObserver in the process() method. When preparing the URL rewrite data for the admin store view, all available store views are prepared instead of only those that have the correct root category (if ($storeViewCode === StoreViewCodes::ADMIN) { and the following lines).

I did some debugging and tried to find a fix myself, but since we're a bit pressed to get those imports working, I opted for a workaround instead. Let me know if you have any questions.

getAttributeSetNameById called on null

I have only specified include_in_menu in the default file and upgraded to 4.0.0-alpha1

{
    "default-values": {
	"catalog_category": {
	    "include_in_menu": null
	}
    }
}

And I am getting

 Uncaught TypeError: Argument 1 passed to TechDivision\Import\Category\Listeners\SortCategoryListener::getAttributeSetNameById() must be of the type integer, null given, called in /home/example/example/projects/shop.example.com/vendor/techdivision/import-category/src/Listeners/SortCategoryListener.php on line 377 and defined in /home/example/example/projects/shop.example.com/vendor/techdivision/import-category/src/Listeners/SortCategoryListener.php:396
Stack trace:
#0 /home/example/example/projects/shop.example.com/vendor/techdivision/import-category/src/Listeners/SortCategoryListener.php(377): TechDivision\Import\Category\Listeners\SortCategoryListener->getAttributeSetNameById(NULL)
#1 /home/example/example/projects/shop.example.com/vendor/techdivision/import-category/src/Listeners/SortCategoryListener.php(176): TechDivision\Import\Category\Listeners\SortCategoryListener->update('Root/Produkte/S...', Array)
#2 [internal function]: TechDivision\Import\Category\Listeners\SortCategoryListener->handle(Object(League\Event\Event), Object(TechDivision\Import\ 

might that be related / is something wrong with the defaults?

Originally posted by @amenk in #66 (comment)

Incorrect rename of url_key

Hi Tim,

I have a problem with categories that shares the same url_key but have a different url_path

Prerequisites:

  1. Magento 2.3.4
  2. PHP 7.3
  3. import-cli-simple 3.8.7

Description:

I think that is legit to have the same url_key for categories that have a different url_path.

For example if i have these categories:

Path Url Key Expected Url Path
Default Category/Linee linee linee
Default Category/Stili stili stili
Default Category/Linee/All Ready all-ready linee/all-ready
Default Category/Stili/All Ready all-ready stili/all-ready

The url key in the last line is automatically converted.
This happens because the import process does not allow to duplicate url_key and doesn't take into consideration the real url_path of the categories.

The conversion is done at these lines:

$urlKey = $this->makeUnique($this->getSubject(), $this->getValue(ColumnKeys::URL_KEY));

$urlKey = $this->makeUnique($this->getSubject(), $this->convertNameToUrlKey($this->getValue(ColumnKeys::NAME)))

and it is done before the url_path is calculated.
Unfortunatly i didn't find a rapid solution so i removed $this->makeUnique for now.

Thanks in advance

CategoryUrlRewriteObserver can't handle renamed categories

Prerequisites:

  • Magento 2.3
  • PHP 7.2
  • composer require techdivision/import-cli-simple:3.1.0-beta9
  • Categories are uniquely identifiable by some external ID, as might be coming from a PIM.
  • The shop contains category A with Name "Blub".
  • The import file contains the same category A but with changed name "Whee".

Description

If categories have some unique external ID coming from a PIM and if we use the category import to continuously update the categories, then the case might arrise that a category changes its name. In our case we have some kind of mapper, which takes the data from the PIM and translates it into a CSV format, which the M2IF category import can understand. So, if category A changes its name from "Blub" to "Whee", we have to include the old name path "[parent category name path]/Blub" in the created CSV file. But in the same row the CSV file will contain the new name "Whee" for the category.

Now the following happens:

  1. In TechDivision\Import\Category\Observers\CategoryUrlRewriteObserver the URL rewrites are written to a new CSV file with prefix url-rewrite. For category A the URL path "[parent category name path]/Blub" will be written together with the URL rewrites to allow identifying the category at a later point.
  2. In the Subject that contains the UrlRewriteUpdateObserver, the categories will be loaded again from the database. The $categories property in TechDivision\Import\Category\Subjects\AbstractCategorySubject will then contain the data of the categories referenced by their name path as it is currently stored in the database. In particular, category A will be referenced by the name path "[parent category name path]/Whee", which contains the new name.
  3. Now TechDivision\Import\Category\Observers\UrlRewriteUpdateObserver process the CSV file with prefix url-rewrite, in which category A is still referenced with the old name path "[parent category name path]/Blub". So when the observer tries to get the data for the category A, it will use the old name path. Then the method getCategoryByPath() in TechDivision\Import\Category\Subjects\AbstractCategorySubject cannot find the category and throws an exception.

Import of Category URL rewrites does not work

Prerequisites:

  • Magento 2.3
  • PHP 7.2
  • composer require techdivision/import-cli-simple:3.1.0-beta9

Description:

In TechDivision\Import\Category\Observers\UrlRewriteUpdateObserver the process() method has a number of problems:

The lines

        foreach ($this->existingUrlRewrites as $existingUrlRewrites) {
            foreach ($existingUrlRewrites as $existingUrlRewrite) {

first loop over the URL rewrites rows and then over the columns.

Within the loop the rewrite data does not set the is_autogenerated field:

                    $attr = array(
                        MemberNames::REDIRECT_TYPE    => 301,
                        MemberNames::TARGET_PATH      => $targetPath,
                    );

The problem here is that the Magento (at least as of EE 2.3) loads category collections by joining the URL rewrites table and filtering by is_autogenerated = 1 instead of filterung by the redirect type. So when a 301-redirect is created that has is_autogenerated = 1 set, the collection loads the same category twice with two different URL rewrites joined. An exception is thrown. Something like "Item with ID ?? alread exists in collection".

I created a patch for this.
bugfix-[5.x]-fix-url-rewrite-update.diff.zip

Functionality

There is also the problem with thUrlRewriteUpdateObserver handles rewrites. It doesn't resolve transitive URL rewrites. And as such it does not handle rewrite loops correctly. I modified the implementation of the UrlRewriteUpdateObserver to handle this.

class UrlRewriteUpdateObserver extends \TechDivision\Import\Category\Observers\UrlRewriteUpdateObserver
{
    protected function process()
    {
        if ($this->getSubject()->getConfiguration()->hasParam(\TechDivision\Import\Category\Utils\ConfigurationKeys::CLEAN_UP_URL_REWRITES) &&
            !$this->getSubject()->getCoreConfigData(\TechDivision\Import\Category\Utils\CoreConfigDataKeys::CATALOG_SEO_SAVE_REWRITES_HISTORY, true))
        {
            // in our client's case we don't have to deal with this case
            parent::process();
            return;
        }

        // since we have our own implementation for this case, we skip the implementation in the immediate
        // parent class
        \TechDivision\Import\Category\Observers\UrlRewriteObserver::process();

        // get new target path for the current category
        $newTargetPath = $this->prepareRequestPath();

        // create redirect URL rewrites for the existing URL rewrites and save the old target path, if any
        $oldTargetPath = null;
        foreach ($this->existingUrlRewrites as $existingUrlRewrite)
        {
            if ($existingUrlRewrite[MemberNames::REDIRECT_TYPE] == 0)
            {
                // in this case, this is the previous main rewrite with
                $oldTargetPath = $existingUrlRewrite[MemberNames::REQUEST_PATH];
            }
            else
            {
                // nothing do to for redirect_type!=0 if the target path is alredy correct
                if ($existingUrlRewrite[MemberNames::TARGET_PATH] === $newTargetPath)
                    continue;
            }

            // if updating the existing URL rewrite would create an identity rewrite, the existing URL rewrite
            // has to be deleted
            if ($existingUrlRewrite[MemberNames::REQUEST_PATH] === $newTargetPath)
            {
                $this->deleteUrlRewrite([
                    'url_rewrite_id' => $existingUrlRewrite[MemberNames::URL_REWRITE_ID]
                ]);
                continue;
            }

            // override data with the 301 configuration
            $attr = array(
                MemberNames::REDIRECT_TYPE    => 301,
                MemberNames::TARGET_PATH      => $newTargetPath,
                MemberNames::IS_AUTOGENERATED => 0
            );
            // is_autogenerated=0 is necessary, since Magento 2 handles category collections in a weird way:
            // instead of searching for redirect_type=0 it filters by is_autogenerated=1

            // merge and return the prepared URL rewrite
            $existingUrlRewrite = $this->mergeEntity($existingUrlRewrite, $attr);

            // create the URL rewrite
            $this->persistUrlRewrite($existingUrlRewrite);
        }
    }

    protected function initializeUrlRewrite(array $attr)
    {
        // iterate over the available URL rewrites to find the one that matches the request path, if any
        foreach ($this->existingUrlRewrites as $urlRewrite)
        {
            // if the request path matches, this rewrite has to be the new main rewrite (i.e. redirect_type=0)
            if ($urlRewrite[MemberNames::REQUEST_PATH] === $attr[MemberNames::REQUEST_PATH])
            {
                // make sure it is not processed any further after inserting
                $this->removeExistingUrlRewrite($urlRewrite);
                return $this->mergeEntity($urlRewrite, $attr);
            }
        }

        // simple return the attributes, so that a new rewrite is created
        return $attr;
    }
}

Use defaults value for include_in_menu to avoid inconsistencies in add_update mode

Magento version: 2.3.4-p2
techdivision/import 16.5.2

We are importing categories via add_update mode.

Background

First we had the column include_in_menu = 1 and everything went fine. Then we decided that we do not want to overwrite that flag in already existing categories and removed the column completely, believing that it defaults to 1 (like when creating a category via admin panel). When we moved from staging to live, we did a full new import and bang: all the categories where not shown.
Worse than that: The category can also not be activated via admin panel any more.

Analysis

A category created via admin panel differs from the imported one in the way, that include_in_menu is completely missing.
Is is not 0, or 1 .. it is just not there.

For the imported category it shows "included in menu = yes" in the admin panel, but in the store front it is not shown in the menu. Also setting it to no and back to yes in the admin panel does not seem to help.

vergleich-test

Possible solution

Set all the values which are omitted as columns to the correct defaults (those Magento uses in admin panel) when creating new category entities.

UrlRewriteUpdateObserver uses wrong delete method on bunch processor

Prerequisites:

  • Magento 2.3
  • PHP 7.2
  • composer require techdivision/import-cli-simple:3.1.0-beta9

Description

In TechDivision\Import\Category\Observers\UrlRewriteUpdateObserver in the deleteUrlRewrite() method, the wrong method removeUrlRewrite() is called on the bunch processor TechDivision\Import\Category\ServicesCategoryBunchProcessor. Instead of removeUrlRewrite() the method is called deleteUrlRewrite().

UrlKeyAndPathObserver creates wrong URL path for non-admin store views

Prerequisites:

  • Magento 2.3
  • PHP 7.2
  • composer require techdivision/import-cli-simple:3.1.0-beta9
  • At least 2 store views A and B within the same website, such that the categories in store view A inherit the URL keys from the admin store view and the categories in store view B have different URL keys to the ones in admin and A. This happens naturally if A and B represent different languages.

Description

In TechDivision\Import\Category\Observers\UrlKeyAndPathObserver in the process() method, the URL path is put together by taking the URL key from the current row and the stored URL path from the existing parent category ($existingCategory[MemberNames::URL_KEY]). The problem is that $existingCategory = $this->getCategoryByPath($categoryPath) contains only the values for the admin store. So, if the categories in store view A inherit the URL keys from admin, then the URL keys for store view A are created correctly. But

  • if the categories in store view B have different URL keys from the ones in admin,
  • and if category Y has parent category X,
  • and if the URL paths of X differ in admin and B,

then the URL path that is created for Y in B will consist of the URL path of X from admin and the URL key of Y from B, which is not correct.

Installation

Good afternoon!

Please tell me how to install the import-category module.

With the use of these commands, nothing happens:
composer config repositories.techdivision git https://github.com/techdivision/import-category
composer require techdivision/import-category

Thanks in advance

Undefined index: techdivision_import_utils_entityStatus_memberName (4.0.0-alpha.2)

getting this error in 4.0.0-alpha2

PHP Notice:  Undefined index: techdivision_import_utils_entityStatus_memberName in /home/user/example/projects/shop.example.com/vendor/techdivision/import/src/Actions/GenericIdentifierAction.php on line 51
	PHP Fatal error:  Uncaught Error: Method name must be a string in /home/user/example/projects/shop.example.com/vendor/techdivision/import/src/Actions/GenericIdentifierAction.php:54
	Stack trace:
	#0 /home/user/example/projects/shop.example.com/vendor/techdivision/import-category/src/Services/CategoryBunchProcessor.php(1058): TechDivision\Import\Actions\GenericIdentifierAction->persist(Array, NULL)
	#1 /home/user/example/projects/shop.example.com/vendor/techdivision/import-category/src/Observers/CategoryObserver.php(411): TechDivision\Import\Category\Services\CategoryBunchProcessor->persistCategory(Array)
	#2 /home/user/example/projects/shop.example.com/vendor/techdivision/import-category/src/Observers/CategoryObserver.php(165): TechDivision\Import\Category\Observers\CategoryObserver->persistCategory(Array)
	#3 /home/user/example/projects/shop.example.com/vendor/techdivision/import-category/src/Observers/AbstractCategoryImportObserver.php(54): TechDivision\Import\Category\Observers\CategoryObserver->process()

Command

 vendor/bin/import-simple import:create:ok-file &&  vendor/bin/import-simple import:categories --debug-mode=1 --clear-artefacts=0 --log-level=warning

I will provide the CSV privately

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.