SMP Logo
SM Publications
Silver Salon Forums - The premier site for discussing Silver.
SMP | Silver Salon Forums | SSF - Guidelines | SSF - FAQ | Silver Sales


The Silver Salon Forums
Since 1993
Over 11,793 threads & 64,769 posts !!

General Silver Forum



Silver Salon internal search
or

REGISTER (click here) How to Post Photos


customtitle open  SMP Silver Salon Forums
tlineopen  General Silver Forum
tline3open  robots and individual content scrapers

Post New Topic  Post A Reply
profile | register | preferences | faq | search

ForumFriend SSFFriend: Email This Page to Someone! next newest topic | next oldest topic
Author Topic:   robots and individual content scrapers
Scott Martin
Forum Master

Posts: 11520
Registered: Apr 93

iconnumber posted 06-01-2017 11:52 AM     Click Here to See the Profile for Scott Martin     Edit/Delete Message   Reply w/Quote
In the past year, both robots and individual content scrapers (also known as: Web scraping, Screen scraping, Web data mining, Web harvesting, or Web data extraction) have gone wild harvesting data from the Silver Salon Forums. I'm not sure this is entirely a bad thing since many of the scrapers point back to the origin of the scraped data (to the SSF site).

Your Google search results will vary depending on what Google knows about you. Here is an example showing 41,400 results of which only 8,670 are actually directly SSF results.

The numbers of visitors to the SSF remains consistently high. Participation (posts) is the lowest ever.

Managing any WEB site such that the content scrapers are defeated is as big a job as keeping the bad guys away. In fact, managing the content scrapers is a bigger job.

The SSF, even with the donation button, hasn't covered one month of operational costs. We greatly appreciate those you who have contributed.

What to do?

Over the next few weeks I will be pondering this. I will have to consider the sorts of things that don't add significant overhead cost or time.

Right now the fastest and easiest thing to do is make some or all of the forums require 'All Access Permissions membership' to even look into a particular forum. For example: to see the posts/threads in the General Forum, members would have to enter their id/password (once per session). I suspect we would leave the New Members' Forums as they are.

???????????

pondering .... pondering ..... pondering

IP: Logged

wev
Moderator

Posts: 4121
Registered: Apr 99

iconnumber posted 06-02-2017 10:27 AM     Click Here to See the Profile for wev     Edit/Delete Message   Reply w/Quote
Bearing in mind that I have only had one cup of coffee so far, it seems to me that while you are trying to encourage greater use, you are making it more difficult to do so. I know next to nothing about robots, scrapers, et al, but isn't that now the nature of the beast? Are you winning the battles, but losing the war?

Since my little project is hosted by Rootsweb, I have no idea or control over what is crawling around in my posted data. I can say that the primary referrer by far is Ecommerce; SMP doesn't make it in the top 10, I am sorry to say. I did a new build a few days ago and was pondering, while uploading around 300 new pages, why I bother. I certainly make no money from it, though I suspect others do, and I can't remember the last time I got a comment or query. Looking at the user logs, it seems to be of little interest institutionally or academically, which is no great surprise these days. If it was not for the pleasure of teasing out the occasional mystery I come across in my rambles, I think I would declare it done.

Such is the fate of the obsessed, I suppose. Now I think I'll have a beer for breakfast.

IP: Logged

asheland

Posts: 935
Registered: Nov 2003

iconnumber posted 06-02-2017 12:56 PM     Click Here to See the Profile for asheland     Edit/Delete Message   Reply w/Quote
WEV I find your silversmiths marks pages very useful and I truly appreciate the effort!

IP: Logged

Scott Martin
Forum Master

Posts: 11520
Registered: Apr 93

iconnumber posted 06-02-2017 01:12 PM     Click Here to See the Profile for Scott Martin     Edit/Delete Message   Reply w/Quote
Thank you WEV.

IP: Logged

wev
Moderator

Posts: 4121
Registered: Apr 99

iconnumber posted 06-02-2017 06:02 PM     Click Here to See the Profile for wev     Edit/Delete Message   Reply w/Quote
I need to stop writing before the sun rises, when doom and gloom still hangs about my head.

IP: Logged

All times are ET

next newest topic | next oldest topic

Administrative Options: Close Topic | Archive/Move | Delete Topic
Post New Topic  Post A Reply
Hop to:


Ultimate Bulletin Board 5.46a


1. Public Silver Forums (open Free membership) - anyone with a valid e-mail address may register. Once you have received your Silver Salon Forum password, and then if you abide by the Silver Salon Forum Guidelines, you may start a thread or post a reply in the New Members' Forum. New Members who show a continued willingness to participate, to completely read and abide by the Guidelines will be allowed to post to the Member Public Forums.
Click here to Register for a Free password

2. Private Silver Salon Forums (invitational or $ donation membership) - The Private Silver Salon Forums require registration and special authorization to view, search, start a thread or to post a reply. Special authorization can be obtained in one of several ways: by Invitation; Annual $ Donation; or via Special Limited Membership. For more details click here (under development).

3. Administrative/Special Private Forums (special membership required) - These forums are reserved for special subjects or administrative discussion. These forums are not open to the public and require special authorization to view or post.


| Home | Order | The Guide to Evaluating Gold & Silver Objects | The Book of Silver
| Update BOS Registration | Silver Library | For Sale | Our Wants List | Silver Dealers | Speakers Bureau |
| Silversmiths | How to set a table | Shows | SMP | Silver News |
copyright © 1993 - 2022 SM Publications
All Rights Reserved.
Legal & Privacy Notices