Documentation
Performance issues

Size mattes, depending on your web server there are limitations on thee amount of items you can have in your database or what size the productfeeds can be. How big really depends on the server.

Feed import limitations

The impot is quite memory efficient, items are parsed one by one, memory should not be an issue van importing feeds.

The main issue when importing datafeeds (using the web interface) is time, time limits also apply to the feed configuration. Often shared web hosts have a time limit of 30 seconds for running web scripts. If this exceeds you will get on error message, an error 500 page or even a blank page.

What might help:

  • Use csv files instead of xml.
  • enable 'curl' as download method, the feeds are fetched first and parsed in a second step. The feeds are cached as well, so if the first import fails the second might succeed since there is no download for the second run. add define('FORCE_CURL','yes'); to your feeds.php
  • download the files manually put them on your server;Use a path instead of a url in the feed configuration, like : /var/www/htdocs/affiliatfeeds.nl/feeds/hotels.csv ( the path depends on your server)
  • during feed configuration limit the number of items in the feed, for example for zanox you can use the 'testmode'. After configuration remove the limitation and run the import from the CLI. If you must run the import from the web web keep the limitation or fetch a queried feed.
  • use ini_set('max_execution_time',120); in your feeds.php ( might by disabled)
  • buy or rent a better server.

Item display limitations.

Most (shared) hosts are not designed to handle large databases. These servers are tuned to handle a lot of small websites. How far you can go really depends on the server, especially memory and mysql  settings.

Reactie toevoegen
 
Callbacks

Generic structure callback functions

function yourcallback_cb(&$item) {
# your stuff here
generic_cb($item);
#more stuff here
}

Next to solving some encoding isues the function generic_cb maps the fields from the feed to the fields in the database.  See this image ( connect the dots ).

So after the call to generic_cb you have the deeplink in href ( $item['href'] ) the fields from 'Field Selection' in menu_1 to menu_9, the group (Select 0) in menu_0, the name of the feed ( = merchant ) in the field feed and the price in prijs

If you insert you code before the call to generic_cb you must use the fields from the datafeed ( name, country etc) if you put your code after the call to generic_cb you must use the database fields ( title, menu_1 etc). This allows to use a single callback function for several feeds regardless the structure of the feed

Excluding items

You can exclude items by clearing the 'title' feed or the complete $item array:

function yourcallback_cb(&$item) {
generic_cb($item);
if ( condition ) {
$item['title']='';
}
}

for condition you can use any (pattern) matching on fields in the item array

examples

'Ugly' ==$item['menu_1']
excludes when Select 1 is exactly Ugly
 
preg_match("/ugly/i",$item['menu_1'] )
 excludes any item having ugly as part the Select 1 field ( case incentive )

 

Changing Values

in the callback you can change any value as you like.

function travel_cb(&$item) {
generic_cb($item);
$item['menu_1']=str_ireplace("England","United Kingdom",$item['menu_1']);
}


Importing offers only

Assuming you have a feed with 'old'-prices and 'current'-prices, and you wish to import the offers only.

Assign the 'current' price to the field 'price' and the 'old' price to Select 9 ( you can pick any free  field)

function offersonly_cb(&$item) {
generic_cb($item);
#be sure your prices are localized ( thus 10.00 and not 10,00)
#otherwise: 
#$item['prijs']=str_replace(',','.',$item['prijs']);
#$item['menu_9']=str_replace(',','.',$item['menu_9']);

if ( $item['prijs'] >=  $item['menu_9'] ) { # this is not an offer
$item['title']=''; #exclude
return;
}
#put the offer in the Select 9 field.
$item['menu_9']= $item['menu_9']-$item['prijs'];

#append the offer to the description 
 
$item['description'].="
You discount ". $item['menu_9']."
";

Cascading functions

similar feeds ofter share similar problems, for example renaming categories. To reuse and share code callback functions can simply call each other. There is just on catch, the function generic_cb must be called once, and just once.

Below a structure for solving this for 'fashion', each 'fashion' feed will call the 'fashion_cb' function ( configured in the feed configuration ) unless it need some special handling having it's own callback ( and the shared one)

 

<?php
# always call generic_cb once
# I use _cb for function that are calling generic_cb, thus in the 
# feed configuration ( joomla admin ) I only have function with _cb

# ensure to use &$item and not $item in the function definition

function fashion_cb(&$item) {
    generic_cb($item);
    fashion($item);
}

#generic fashion script
function fashion(&$item) {
    #do fashion stuff
    $item['menu_4']=str_ireplace("woman","women",$item['menu_2']);

    #etc etc

}
#special callbak for easyunderwear feed
function easyunderwear_cb(&$item) {
    generic_cb($item);
    if ( $item['stock']==0 ) {
    $item['title']='';
    return;
}
    fashion($item);
}

 

 

 

 

 

Reactie toevoegen
 
Using Netaffiliation datafeeds

Netaffiliation productfeeds ( Catalogues ) are supported. The  feeds can be found under 'Tools' in the webmaster area of the Netaffiliation website.

When using CSV feeds you need to customize the name of the 'description' field. The default name is 'descriptif' change this to 'description' using the table on the catalogues page.

For XML feeds, a XML parser for Netaffiliation is avaible from version SVN:1062, this parser renames the descriptif field and explodes the 'rubrique' ( category path). For older version the class 'Tradetracker' can be used, however this parser will not rename the descriptif or explode the path.


The tools page also depicts a List of your catalogues this feed of feeds is not yet supported

Reactie toevoegen
 
Share a sale product datafeeds

The shareasale network provides CSV (piped) formated datafeeds. Adding the feeds to the component could be very straightforward, unfortunately the share a sale feeds have some issue's:

 

  • they don't have a header row
  • the are zipped instead of gzipped ( must be windows people over their)
    • using ftp access the files are available gzipped
  • the feeds can not be downloaded (easily) automaticly.
  • the share a sale feed does not have your ID in the feed

all of these can be solved ( requiring datafeeds version 1035)

Shareasale.com Affiliate Software

Missing header row

The share a sale feeds have a fixed format and order of the fields, althought this is fine for a feed from a single network the CSV parser is created generic for all network, thus requiring a header row ( different for each network).

Zipped format

unlike the gzip format the zipped format is might not be integrated in your version of php . And php does not support a 'on the fly' parsing ( gzopen) of zipped files. Making the operation more memory savy

Automatic download

It is possible to request ftp download for datafeeds, however this requires approval of each merchant. Workaround is to use a CURL login and fetch script.

Manual fetch and parsing

Adding a parser

(this step is required once)

at the bottom of this page their is a link to download the share a sale parser. ( SVN:1035)

Put the contents of the script into /administrator/components/com_datafeeds/cron/xml_custom.inc

create the file if it doesn't exist. In the feed configuration (joomla administrator) you should see the 'Share a Sale' parser.

Adding your user ID

In the script just added you will find a function shareasale with the line

$item['href']=str_replace("YOURUSERID","435017",$item['href']);

replace the number. (there are a few reasons this is not in the parser and it is done this way, donate a few beers and we can chat about it)

Manual download

(this step is required each time you want to refresh the feed)

visit the share a sale website and download the datafeed. unzip the file an upload the resulting text file to your system. The location is not important, for example

/PATH TO YOUR JOOMLA ROOT/feeds/merchantid.txt

After each download import ( cron ) the data into the component.

Feed configuration

(this step is required once)

In the feed configuration add a new feed, select the share a sale parser. In the feed url you can put a web-url pointing to the feed

http://[your website]/feeds/merchantid.txt

however it is far more efficient to use a file pointer

/PATH TO YOUR JOOMLA ROOT/feeds/merchantid.txt

no clue where your JOOMLA ROOT is? check the Global Configuration> system > path to log folder. This is mostly correct, replace logs by feeds/.txt and you got your path.

In the callback function replace the generic_cb by shareasale_cb

Fetching using ftp access

If a merchants grants ftp access to his datafeeds you can simply use the ftp path to fetch the data, for example feed url:

ftp://[your username]:[your password]@datafeeds.shareasale.com/11155/11155.txt

 

Automatic fetch and parsing using http access

Yes is is possible to use curl to fetch the feeds using http.

This is available on request and please make a donation before asking.

Downloads

share a sale parser class

demo config ( screenshot)

Eat the pudding (example/demo)

Reacties lezen (2)
 
CSV files

The component can import any CSV formatted file delimited with tab, ',', ';' or '|' and enclosed with '"' (optional). The CSV files must have a header row:

category;sub_category;deeplink;image;title;description
Trousers;Men;http://bladibla.com;httm://bladibal.com/a.jpg;Nice Jeans;bla di bla
T-shirts;Girls;http://bladibla.com/1.html;httm://bladibal.com/b.jpg;Nice too;bla di bla

 

The bare minimum for a productfeed is a title, however this is not a really useful feed.

Not that the 'auto detect' for the parser class in the feed configuration does not work for CSV files, select a parser manually.

Several program's provide XML feeds each type of XML formatted feed requires a dedicated parser. However one of the existing XML parsers might cover your feed. Otherwise you need to create your own parser

 

Reactie toevoegen
 
Meer artikelen...
<< Start < Vorige 1 2 3 4 Volgende > Einde >>

Pagina 1 van 4