SPECIAL OFFER: Only $59 for access to everything in OSTraining for 1 year! You save $85!  1
Join today and get access to 1,000's of books and videos. Learn WordPress, Drupal, Magento, Joomla and more! Sign up today!

RSS vis-a-vis other methods

3 years 5 months ago #123415 by iqsafe
RSS vis-a-vis other methods was created by iqsafe
I used to use RSS to get information from other sites into mine. Take for example my website that I made a few years ago Baghdad Business Center . If you look at the right side bar you will see I brought business news into my site by using the RSS. My question here: If there is no RSS on the site that I want to fetch information from into my site, are there other ways to do so? I intend to bring information of job vacancies from the webpage NCCI vacancies into my new site Suquna . Thanks.

Please Log in to join the conversation.

3 years 5 months ago #123418 by Valentin
Replied by Valentin on topic RSS vis-a-vis other methods
Hi iqsafe,

If the site doesn't allow an easy way to fetch their information with JSON or RSS, you may consider to build an scraper with PHP.

Here is an example: www.ostraining.com/blog/coding/extract-image-php/

Being honest, is not an easy one, since requires to be very good in regular expressions .

Kind regards,
Valentín

Follow us on Twitter - twitter.com/OSTraining
Like us on Facebook - facebook.com/ostraining
The following user(s) said Thank You: iqsafe

Please Log in to join the conversation.

3 years 5 months ago #123422 by iqsafe
Replied by iqsafe on topic RSS vis-a-vis other methods
Thank you so much. At least I have known that this is possible but not quite easy.

Please Log in to join the conversation.

3 years 5 months ago #123427 by Valentin
Replied by Valentin on topic RSS vis-a-vis other methods
You're welcome, iqsafe!

Kind regards,
Valentín

Follow us on Twitter - twitter.com/OSTraining
Like us on Facebook - facebook.com/ostraining
The following user(s) said Thank You: iqsafe

Please Log in to join the conversation.

3 years 5 months ago #123506 by iqsafe
Replied by iqsafe on topic RSS vis-a-vis other methods
Thank you again, I have seen the your example on scraping. I have a question here please:

1- Is there a good tutorial/link that teaches good regular expression. I studied it long time ago but I am rusted.
2- If you are searching for the img tag and then the src and then you store these srcs as array in the $match variable, how about the img, can you please explain a bit more about the regular expression.

3- Is there an example about scraping links (for example the job links)

Thanks

Please Log in to join the conversation.

3 years 5 months ago #123518 by Valentin
Replied by Valentin on topic RSS vis-a-vis other methods
1. We don't have material about the topic, besides recent examples like the one I shared before to crawl images. Take a look to this site: www.regular-expressions.info/

2. It's part of the answer above.

3. Something like this may works as base:
preg_match_all( '|<a.*?href=[\'"](.*?)[\'"].*?>|i',$html, $matches );

Follow us on Twitter - twitter.com/OSTraining
Like us on Facebook - facebook.com/ostraining
The following user(s) said Thank You: iqsafe

Please Log in to join the conversation.

3 years 5 months ago - 3 years 5 months ago #123519 by Nick
Replied by Nick on topic RSS vis-a-vis other methods
Hi iqsafe,

Before heading in that direction, please double check with the site owner if it's OK for you to use their content, since it's copyrighted by default.

That being said:
1) We don't have any classes on regular expressions yet, but when I was learning, www.regular-expressions.info/tutorial.html was an excellent resource that I found.

2) It's not necessary to store the img tag in the array, since it's generic and can be recreated later when needed. For example, here's how Valentin recreates it in the tutorial:
echo '<img src="http://www.website.any' . $matches[ 1 ][ 0 ] . '" />';

So the important part is the unique URL.

3) You'd use the same approach, except instead of "<img.*?src=", you'd use "<a.*?href="

Hope this helps!

Kind regards,
Nick

Follow us on Twitter - twitter.com/OSTraining
Like us on Facebook - facebook.com/ostraining
The following user(s) said Thank You: iqsafe

Please Log in to join the conversation.

3 years 5 months ago - 3 years 5 months ago #123705 by iqsafe
Replied by iqsafe on topic RSS vis-a-vis other methods
1- Thank you for your input on this topic. At this stage I am only learning and refreshing my some skills on regular expression, I am not really intending to include content not unless there is a an RSS on the concerned site.

2- I have a question here, I take news from the site Iraq Business News into my website Baghdad Business Center by using its provided feed. The strange thing now that when I go to that website I cannot see the RSS feed icon. I am also looking for other feeds for other menu items on that site Iraq Business News . Do you think it is there but invisible somehow. Thanks.

Please Log in to join the conversation.

3 years 5 months ago - 3 years 5 months ago #123706 by iqsafe
Replied by iqsafe on topic RSS vis-a-vis other methods
Thanks a lot Valentine for providing a hint of a regular expression which was very interesting to look into and test. The expression you provided did not work, it only gave me a blank screen.

Following is part of flowed output produced as follows:

www.facebook.com/USEmbassyBaghdad
twitter.com/USEmbBaghdad
www.youtube.com/user/usembassybaghdad
/multimedia.html
/galleries.html
/podcast.html
#skipBreadcrumb
#skipLeftNav
/about-us.html
/ambassador.html
/officesdepartments.html
/american-iraqi.html
/holidays.html
/jobsvacancies.html
/gso_procurement.html
photos.state.gov/libraries/iraq/5/jobs/v...technician-erbil.pdf
photos.state.gov/libraries/iraq/5/jobs/v...eman-fsn-7-erbil.pdf
photos.state.gov/libraries/iraq/5/jobs/VA 16-e-07-work-order-clerk-erbil.pdf
photos.state.gov/libraries/iraq/5/jobs/v...rity-guard-erbil.pdf
photos.state.gov/libraries/iraq/5/jobs/v...ty-guard-kurdish.pdf
BaghdadJobs@state.gov
BaghdadJobs@state.gov
javascript:window.print()
javascript:mailpage();


Apparently there are jobs links in between. Here is a question:

On the RE you provided I noted there is a (.*?) after the (<a) tag. I suppose that ant (a) tag should be followed by a space (\s), why you followed the said tag with the (.*?) (parentheses not counted it is only for clarifications.

Thanks a lot

Please Log in to join the conversation.

3 years 5 months ago #123708 by Valentin
Replied by Valentin on topic RSS vis-a-vis other methods
Hi iqsafe,

Look in the source code for "rss" references, probably they have a link to the feed in there.

On the RE you provided I noted there is a (.*?) after the (<a) tag. I suppose that ant (a) tag should be followed by a space (\s), why you followed the said tag with the (.*?) (parentheses not counted it is only for clarifications.

By using (.*?) you will crawl any character.

Hopes this clarification helps

Kind regards,
Valentín

Follow us on Twitter - twitter.com/OSTraining
Like us on Facebook - facebook.com/ostraining
The following user(s) said Thank You: iqsafe

Please Log in to join the conversation.

3 years 4 months ago - 3 years 4 months ago #125919 by iqsafe
Replied by iqsafe on topic RSS vis-a-vis other methods
I have read Ostraining tutorial (Showing an RSS Feed in a Drupal Block). How to further manipulate this. I mean for example if one wants to put only bulleted titles in a block. Is that possible through a view and how ?. I very appreciate your kind help. I have done this before with Drupal 6 but not 7 which I am using now.

Please Log in to join the conversation.

3 years 4 months ago #125929 by Nick
Replied by Nick on topic RSS vis-a-vis other methods
Hi iqsafe,

Sure thing. Please give our "Creating RSS Feeds in Drupal Using Views" tutorial a try:
www.ostraining.com/blog/drupal/creating-...-drupal-using-views/

Hope this helps! Let us know if you have any questions and we'll be glad to answer.

Kind regards,
Nick

Follow us on Twitter - twitter.com/OSTraining
Like us on Facebook - facebook.com/ostraining

Please Log in to join the conversation.

3 years 4 months ago #125960 by iqsafe
Replied by iqsafe on topic RSS vis-a-vis other methods
Thank you Nick for your reply. In fact I did not mean creating RSS, I meant how to use RSS in filling my website. Thanks

Please Log in to join the conversation.

3 years 4 months ago #125986 by Nick
Replied by Nick on topic RSS vis-a-vis other methods
Hi iqsafe,

OK, for that part, you're going to want to give Aggregator a look:
www.ostraining.com/blog/drupal/aggregator/

Could you give that a try and see if it will work for you? If you'll need additional customization, Steve or Daniel will be able to help once they are back in the office.

Kind regards,
Nick

Follow us on Twitter - twitter.com/OSTraining
Like us on Facebook - facebook.com/ostraining
The following user(s) said Thank You: iqsafe

Please Log in to join the conversation.

3 years 4 months ago #125994 by iqsafe
Replied by iqsafe on topic RSS vis-a-vis other methods
Thanks for sending a tutorial. I have have known how to do it but I have facing a problem. I visited the Site and I tried to take all the 52 posted jobs from the RSS URL
www.bayt.com/live-bookmarks/iq-rss.xml but I found that I only get 11 posted jobs. Is this a wrong feed or I might be missing something. Thanks a lot.

Please Log in to join the conversation.

3 years 4 months ago - 3 years 4 months ago #125995 by iqsafe
Replied by iqsafe on topic RSS vis-a-vis other methods
Thanks for sending a tutorial. I have known how to do it but I have been facing a problem. I visited the Site and I tried to take all the 52 posted jobs from the RSS URL
www.bayt.com/live-bookmarks/iq-rss.xml for the jobs of Iraq, in order to post on my site, but I found that I only get 11 posted jobs. Is this a wrong feed or I might be missing something. Thanks a lot. The list of feeds I took Iraq feed from is www.bayt.com/en/rss/

Please Log in to join the conversation.

3 years 4 months ago #125998 by Daniel Pickering
Replied by Daniel Pickering on topic RSS vis-a-vis other methods
Hi iqsafe,

I think that feed is just for the latest jobs. This one is the all jobs feed I believe.
www.bayt.com/live-bookmarks/all-rss.xml

Thanks
Daniel
The following user(s) said Thank You: iqsafe

Please Log in to join the conversation.

3 years 4 months ago #126001 by iqsafe
Replied by iqsafe on topic RSS vis-a-vis other methods
A lot of thanks. But can this be filtered on a country with a view?

Please Log in to join the conversation.

3 years 4 months ago #126002 by Daniel Pickering
Replied by Daniel Pickering on topic RSS vis-a-vis other methods
When you pull in the data is country it's own field in a view ? If it is you should be able to use contextual filter to sort a country it have it user selectable.

Thanks
Daniel
The following user(s) said Thank You: iqsafe

Please Log in to join the conversation.

3 years 3 months ago #126042 by iqsafe
Replied by iqsafe on topic RSS vis-a-vis other methods
Are you asking me whether country is one of the fields in that feed ? In fact the view is a one that I think I should build, but am I supposed to know whether the country field is part of the feed?

Also can please expand more about contextual filter, I am really very interested to know. Do you have a tutorial.

Thanks in advance

Please Log in to join the conversation.

3 years 3 months ago #126045 by Daniel Pickering
Replied by Daniel Pickering on topic RSS vis-a-vis other methods
Hi iqsafe,

Contextual filters come under the advanced views options we have a class on them here.
www.ostraining.com/class/contextual-views/

Thanks
Daniel

Please Log in to join the conversation.

Join today and get access to 1,000's of books and videos. Learn WordPress, Drupal, Magento, Joomla and more! Sign up today!