Raid Script Commands

Built-in Functions

DOWNLOAD_PAGE

Usage:

This command is used to download a page from a URL in list of URL's extracted. When the URL list is empty, START_URL is used.

DOWNLOAD_NR_PAGE

Usage:

This is an extension to the DOWNLOAD_PAGE command. The difference of this command and DOWNLOAD_PAGE is that, in this command, before download a page, it is check if this is downloaded earlier. This is used in NewsRaider mode and if used in other modes, it ignores this duplicate checking.

GET_LINKS LIST_X

Usage:

This command will extract URL's from the currently downloaded page and add them to the URL list specified. (If no list is specified, default list, LIST_1 is used.) Rules specified in 'Quantifiers' are used for URL extraction. Extracted URLs are stored in LIST_X where X is any number from 1 to 10. If the list is not empty, URLs are added to the end of the list.

Examples

GET_LINKS LIST_3

GET_LINKS LIST_X

Usage:

This command will extract URL's from the currently downloaded page and add them to the URL list specified. (If no list is specified, default list, LIST_1 is used.) Rules specified in 'Quantifiers' are used for URL extraction. Extracted URLs are stored in LIST_X where X is any number from 1 to 10. If the list is not empty, URLs are added to the end of the list.

Examples

GET_LINKS LIST_3

FIND_LINE Atext$

Usage:

This command will find a line in the page that contains the text string(s). The line will then be made current and can be copied to the VAR. FIND_LINE can be used with the AND operator.

FIND_LINE is the only way you can change the contents of the LINE input variable thus LINE will contain its text until FIND_LINE is used again or the article is processed.

Examples

FIND_LINE "document.write"

VAR = LINE

//. commands to process VAR

FIND_LINE "document.write" AND "date"

FIND_LINE "<b>" AND "</b>" AND "FONT"

VAR_FIND_LINE Atext$

Usage:

This command will find a line in the VAR that contains the text string(s). The line will then be assigned to VAR. VAR_FIND_LINE can be used with the AND operator.

Examples

VAR_REPLACE Atext$ WITH Btext$

Usage:

This command is the one you will be using most in your raid scripts. It operates on VAR and replaces the first occurrence of Atext$ with Btext$.

Examples

VAR_REPLACE "January" WITH "01"

VAR_REPLACE_ALL Atext$ WITH Btext$

Usage:

Unlike VAR_REPLACE which operates on the first occurrence of Atext$ this function will replace all occurrences of Atext$ with Btext$

Example

VAR_REMOVE Atext$

Usage:

This command will remove the first occurrence of Atext$ in VAR.

Example

VAR_REMOVE "<!No Formatting>"

VAR_REMOVE_ALL Atext$

Usage:

This command will remove the all occurrences of Atext$ in VAR.

Example

VAR_REMOVE_ALL " ,"

VAR_REMOVE_FROM Atext$ TO Btext$

Usage:

This command will remove from all text beginning with Atext$ until the end of the first occurrence of Btext$.

This command has two special controls you can use if needed, these are VAR_START and VAR_END

Example

VAR_REMOVE_FROM "<div" TO "/div>"

VAR_REMOVE_FROM VAR_START TO "<title>"

VAR_REMOVE_FROM_ REVERSE Atext$ TO Btext$

Usage:

This command will remove from all text beginning with first occurance of Atext$ when searched from the end of VAR, until the beginning of the first occurrence of Btext$ reverse searched again from the position of Atext&.

This command has two special controls you can use if needed, these are VAR_END and VAT_START.

Example

VAR_REMOVE_FROM_REVERSE "/div>" TO "<div"

VAR_REMOVE_FROM_REVERSE VAR_END TO "</title>"

VAR_REMOVE_FROM_REVERSE "<tr> TO VAR_START

VAR_REMOVE_ALL_FROM Atext$ TO Btext$

Usage:

This command will remove every instance of text beginning with Atext$ until the end of the next occurrence of Btext$. It is identical to VAR_REMOVE_FROM in function except it will remove every occurrence of Atext$ to the next Btext $.

Example

Instead of using 3 times:

VAR_REMOVE_FROM "<div" TO "/div>"

VAR_REMOVE_FROM "<div" TO "/div>"

VAR_REMOVE_FROM "<div" TO "/div>"

You can use once:

VAR_DELETE_AT Position%

Usage:

This command will delete the contents of the VAR from Position% to end of the string. Position can be negative or positive. If negative, the position is count from the end of the string.

Note: Position % cannot be 0

Example

VAR_DELETE_AT 5

VAR_INSERT_AT Position% Atext$

This command will insert the Atext$ at the position in VAR specified by Position%. If Position% is a VAR_START or VAR_END number then the insertion takes place from the absolute start or end of VAR respectively other wise from the beginning.

Note: Position % cannot be 0

Examples

VAR="HelloWorld"

VAR_INSERT_AT VAR_START "A Big "

"A Big HelloWorld"

VAR_INSERT_AT 1 " Very"

"A Very Big HelloWorld"

VAR_INSERT_AT -5 " "

"A Very Big Hello World"

VAR_INSERT_AT VAR_END "To Everyone"

VAR_MONTH_NUM

Usage:

This command will replace all texts for months to the corresponding month of the year within VAR. So "Jan" and "January" will become 01 and so forth.

DATE_FROM_VAR ".date format mapping.."

Usage:

This command will set the output variable, DATE according to the contents of VAR and the specified date format mapping.

The date format mapping is where you specify using the terms:

MONTH

DAY

YEAR

HOURS

MINUTES

VAR, at the time of using DATE_FROM_VAR, must have each mapping item separated by delimiters that are specified in the mapping.

Examples

DATE_FROM_VAR "MONTH DAY YEAR HOURS MINUTES"

DATE_FROM_VAR "DAY MONTH YEAR,HOURS:MINUTES"

CATEGORY_AUTO_MAP_FROM_VAR

This powerful command will allow you to easily assign PRIMARY_CATEGORY, SECONDARY_CATEGOY and REGION output variables based on the contents of VAR. If you look at the text file CategoryMasterMapping.txt you will see many lines such as:

"tennis" PRIMARY_CATEGORY="Sports",SECONDARY_CATEGORY="Tennis"

"uk_news" REGION="UK"

"sport2" PRIMARY_CATEGORY="Sports"

"allpolitics" PRIMARY_CATEGORY="Politics"

"HEALTH/diet.fitness" PRIMARY_CATEGORY="Health", "SECONDARY_CATAGORY="Diet and Fitness"

This is saying that, given that VAR contains the text at the start of the line the category information for this article should be defined by the reminder of the line. So, if VAR contains "allpolitics" at the time of calling this command then the article will have Politics as its primary category.

STORE_VAR_1 . STORE_VAR_10

Usage:

This command will save the contents of the variable VAR in a temporary storage locations 1 to 10, which can be retrieved by using the command RETRIEVE_VAR_1 to RETRIEVE_VAR_10.

Examples

VAR = BODY

VAR_REMOVE_FROM <h1> to </h1>

STORE_VAR_5

RETRIEVE _VAR_1 . RETRIEVE _VAR_10

Usage:

This command will retrieve the contents stored in temporary storage locations 1 to 10. Currently this command can be used with the following commands.

• VAR
• VAR_INSERT_AT
• IF_VAR_CONTAINS,
• ELSE_IF_VAR_CONTAIN,
• IF_VAR_NOT_CONTAINS,
• ELSE_IF_VAR_NOT_CONTAIN

Examples

VAR = BODY

//.. some commands here to process contents of VAR

STORE_VAR_3

VAR = PAGE

//.. some commands here to process contents of VAR

VAR_INSERT_AT VAR_END RETRIEVE_VAR_3

TITLE = VAR