How to Speed Up Import Process in Store Manager for Magento
In order to speed up the import process you need to check the following options at the last step of Import Wizard:
Select "Add Only" option in case you want to update your store only with new products without changing the existing ones.
When this option is used, import procedure will not check if product already exists but will always add new one. If you perform import from the same file again and again it will create new products each time so there will be duplicated entries. This option should be used only for initial import when you just starting new store. It should NOT be used for regular updates such as automated import.
Check "Cache import data" checkbox that will help to speed up the process if you have slow direct connection.
This option may boost performance when importing from large .csv files (approx. 5000 products and more). For smaller files it may not be effective. Also the more products you already have at your store the more time it will take to create local product cache so if you have 100000 or more products in your store database and want to import about 5000 products from .csv file this option may not be so effective. This option has no effect at all when you're using bridge connection.
Check "Accelerate import by increasing memory usage" checkbox.
It is recommended to have this option always checked unless you have really low amount of memory available at your system. However, in some cases this option may have the opposite effect. It does retrieve ALL products from your store and creates memory maps allowing import procedure to determine faster whether current product is new or already exists. In case if you have large database (100 000 products and more) it may take too long to retrieve all of them and will take too much memory as well. This option is most effective in case of you have up to 10 000 products in your database and want to import a few more thousands.
Uncheck "Reindex product data" checkbox and make full Reindex data in the Magento admin after the import will be completed.
Product import procedure consists of two major parts - product import itself and product indexing. Indexing takes about 60-70% of total import time so when this option is unchecked it may be significantly faster. Nevertheless product indexing is VERY important in Magento. When index is incorrect product may not be visible at your store or may cause various errors. You have to rebuild product index in any case when import is finished. In some cases it may take less time to do that in Magento admin area: System -> Index Management.
- Important images settings: Starting from version 1.9.x Magento requires unique image names for all the images. For the mentioned purposes, in Store Manager were added option to regenerate image name if the same already exists in database. However, since this option checks all the folders with images at your FTP, import might take more time than expected. If you completely sure that your image names are unique you can uncheck this option to speed up the upload.
Please check the screen-capture of all the steps you have to perform:
Other things that may affect import speed:
- Number of Store Views you have checked to import products to. In general, every Store View has its own set of product attributes. If you select two store views import procedure will actually post data twice and respectively three times for three selected Store Views and so on.
- Number of product attributes in attribute set. The more attributes product has the slower import will be. Do not create product attributes that you do not need and delete those ones you're not using. If product attribute has "Use in Layered Navigation" option set to "Yes" it will be included in indexing and thus slow down import and other operations with a product even more.
- Number of .csv fields assigned to product database fields i.e. number of columns you want to import from .csv file to your store. The more columns you have linked the more time it will take to import product. The number of assigned columns also affects product indexing.
- The number of products you already have in your database. Even if you want to import 10 products but already have one million of them in store database it will take long time since import has to perform several checks for every product and it takes significant period of time in case you have large database. If you're starting with empty database at the beginning it does import 20-30 or even more products per second. Every new product added to database will slow down import speed so it may drop to 1 product per second and that is NORMAL.
- Database server performance. Store Manager sends SQL commands to the server. It may take some time for the server to perform these commands, the more CPU and memory resources available at the server the faster it will execute requested command and the faster it will work in general. Do not expect high performance if you're using shared public hosting. Usually, they provide resources for simple HTML websites but not for e-commerce engines.
Generally, you have to remember - large amount of data will always take large amount of time to be processed.