Importing CSV feed

[harvey]

I know that http://www.pricedemon.co.uk/ uses MonkeyCMS with a few bells and whistles, and I was just wondering if any of the extra functionality will ever be available either as a plugin or rolled out in future versions?

I'm looking at a site for a car dealer who has an existing data feed of used stock that gets out to Autotrader etc.

The feed could be ftp'd at regular intervals to the sites server in a zip file. The zip would contain a csv file of the vehicle details, along with multiple images for each vehicle as jpgs named with the vehicle reg no appended with a digit.

If you were to make any of your modifications for Price Demon available Colin, would they be of use to my application?

If not, I'm guessing I'd need to create some sort of script that is scheduled to clear all existing vehicle details, unzip the file and then import the csv into the database? Does that sound feasible?

[Colin]

Price Demon uses XML calls to a third-party price comparison engine - so at present doesn't quite do what you need...

HOWEVER, the mobile comparison section DOES take a CSV feed (a few times a day) and then populates a database so it is doable, if a little complicated. It's a batch of separate scripts that sits on top of Monkey and is obviously very custom stuff to what we current support.

Monkey does already support the import of CSV feeds though - so it might be that you could build on top of that?

[harvey]

When you say the mobile comparison populates a database from a feed, do I take it that it's a separate database to the main MonkeyCMS database, and if so is that the route I should be looking at taking?

I'll have a play, and I'll take a look at the existing CSV import facility too.

I've managed to cobble together a simple standalone php page which displays a list of vehicles from the CSV, but I presume I need to import into a database to be able to display vehicle specific pages with images etc.

[Colin]

It is a separate database - purely for the reason that hte data import is server intensive and I didn't want to cause problems for the main site.

The current import code should be able to take a file, pull it into MonkeyCMS as standard content and then you can do what you want with it. HOWEVER, I'm not sure this would be the most efficient option - I'd consider using it as a basis for your own import on separate tables that are specifically designed for the CSV data and then just put together some basic functions to pull the data from the database and do what you need with it.

[harvey]

Thanks a lot for your advice Colin, I appreciate it's outside of the scope of what you're doing with MonkeyCMS.

I'll let you know how I get on!

[harvey]

I've just PM'd you regarding this Colin

[Colin]

It's not entirely off my radar I'm looking at the potential of moving away from the third-party price comparison bits reasonably soon as it takes away about 20% of the Price Demon income so makes sense to have our own alternative.

Obviously, when this kicks off I'll be doing loads of CSV importing and tidying!

[harvey]

I've been having a play and I've come up with a php script that imports the contents into a definition:

csv_to_monkeycms.txt

It seems to do the job, but I'm just concerned as to whether there might be any security issues I don't know about?

Also, when the csv gets uploaded, images for the cars get uploaded to the same directory. Could you give me any pointers on how I can use these?

[chrisjlocke]

You use $line in the foreach but $arr[] in the SQL string - is that correct?

[harvey]

Sorry Chris, attached the wrong file, here is the actual script.

csv_to_monkeycms.txt