Nata1 Walk THROUGH

Then after the close of your systemWeb section () but before the close of your configuration section () you should add the following ... editing the file path (My implementation does not use this), site URL, SQL connection string, and the Google Key (if applicable). I have removed the quotes from the sections you need to edit, so if you forget one your application will tell you your .config is invalid ... you need to replace all of my square brackets with quoted strings .

>

<-

->

And finally you need to add the exception management section adding your email address and a valid path where ASP.NET has permissions to write a file for the error log (this is also in SQL so you do not really need it)

Now Things Would Be Working, But for A Few Bus in The Spider ... so lets fix The spider weforne iplement it on our site. Open the file engine / indexing / indexutility.cs

First we need to up the timeout on the HttpWebRequest Open the Engine / Normalization / SitePageUtility.cs ... and around line 342 in the GetPage function, you will see a line that looks like this:. Wr.Timeout = 1;

Change It to 10000 AS IT IS in Milliseconds, and 1 Will Cause All Spider Pages To Timeout.

We are going to need to change the default behavior of the spider. It appears to have been designed to follow only relative links, which kept it on the specific site ... For .Text all of the links are absolute, so we need this utility to recognize and spider all links that are on the same domain as our base site. This behavior can be changed in the file Engine / Indexing / IndexUtility.cs. Update the buildSiteURLs function ... Actually just replace it with (careful on the word wrap the editor Really Messed Up this function:

Private static void buildsiteURLS (String URL)

{

// _ Rawpages = new hashtable ();

// for debugging -

// this function will be just like get page URLS

// 1. get the links for a page this don't appear in the root arraylist

// 2. Add the unique urls to the root arraylist

// 3. for Each Unique URL, Recursivly Call The Function

// this function will eventually only return "unique" Page Urls

// OK, Capture All the Urls on the page

Regex rx = new regex ("href // s * = // s * (?: /" (? | (? // s ) ",

Regexoptions.ignorecase | RegexOptions.comPiled;

// regex rx = new regex ("href // s * = // s * (?: (?: /////"] *) /// ") | ( [^ // s] *) "", regexoptions.ignorecase | regexoptions.compiled;

// String hrefpattern = "<(a | a) // s {1}. * href = /" (? [^ / "] ) /" "

ArrayList PageResults = new arraylist (); // regex rx = new regex (HREFPATTERN);

Matchcollection mc;

Try

{

// perform the regex match

String PageText = NULL;

Try

{

PageText = SitePageUtility.getPage (URL);

}

Catch

{

// ok we've logged this

}

IF (PageText! = NULL)

{

_Rawpages [URL] = PageText;

}

Else

{

Return;

}

// system.diagnostics.eventlog log = new system.diagnostics.eventlog ("vivacoStarica");

// log.writeentry ("ERROR Requesting -" URL);

// Log.close ();

MC = rx.matches (pagetext);

Foreach (Match M IN MC)

{

Group g = m.groups ["href"];

// prepare the URL

String linktest = g.value.toupper ();

//if (LinkTest.indexOf ("/") !=0 )continue; no need to check for relative vs static

IF (LinkTest.indexof ("/") <- 1) Continue; // Use this to strip anchors and some javascript

IF (LinkTest.indexof (". JPG")> - 1) Continue;

IF (". GIF")> - 1) Continue;

IF ("LinkTest.indexof (". PDF ")> - 1) Continue;

IF (LinkTest.indexof (". CSS")> - 1) Continue;

// int Testint = linktest.indexof (". Jpg");

String href = g.value;

IF (_debug == true)

Logutility.logevent (entrytype.info, datetime.now, "href: href);

IF (href.indexof ("/") == 0)

HREF = SiteUtility.getsiteBaseURL () href.substring (1); // only append the base if it is a relative link

String Uri = href.split (char.parse ("#")) [0] .tostring (); // Strip any anchor tags

// String defaultPage = getSitedEfaultPage ();

// Add support for Different ExtensStion

if (! _ siteUrls.Contains (uri) && uri! = SiteUtility.GetSiteBaseUrl () && uri! = (SiteUtility.GetSiteBaseUrl () SiteUtility.GetSiteDefaultPage ()) && uri.IndexOf (SiteUtility.GetSiteBaseUrl ()) == 0) {

//if (_debug==true&&&_siteurls.count "6||_debug==false)

{

_SiteURLS.Add (URI);

PageResults.Add (URI);

}

}

Else IF (_debug == true)

{

// log why the url was not added

IF (_SiteURLs.Contains (URI))

Logutility.logevent (entrytype.info, datetime.now, "the url haas already been added to the list";

IF (Uri.indexof (SiteUtility.getsiteBaseURL ())! = 0)

Logutility.logevent (entrytype.info, datetime.now, "the url is off tasmp";

IF (URI == SiteUtility.getsiteBaseURL () || URI == (SiteUtility.getsiteBaseURL () SiteUtility.getsiteDefaultPage ()));

Logutility.logevent (entrytype.info, datetime.now, "the url is for the default page, and is already indexed";

}

}

Logutility.logevent (entrytype.info, datetime.now, "regex url matches for this page:" mc.count.toString ());

Logutility.logevent (entrytype.info, datetime.now "PageResults for this page:" PageResults.count.toString ());

// Now for Each Unique Link ON this page make a recursive call

// Testing - Make Sure We Only Have Unique Urls

}

Catch (Exception EX)

{

Logutility.loGerror (entryType.RecoverableError, DateTime.now, "Error Processing Page" URL, EX);

// throw new applicationException ("Error Occured Building Site Urls", EX);

}

// if (_debug == true)

// if (this.siteurls.count> 5) Return;

FOREACH (String S in PageResults)

{

BuildsiteURLS (s);

}

}

While we are here there is a bug in the SearchResults Repeater designer, I did not find it because I wanted to implement the whole thing as a class in my application directly and avoid all of the carefully crafted but not documented server controls. But I devised this workaround ... There is a problem with the ItemTemplate object, but without it your repeater wont return results ... the solution is to use the alternatingItemTemplate ... So in the file UI / Common / BaseRepeater.cs find the line Around 460 That Reads: // Don't Do Anything If No ItemTemplate

IF (_ItemTemplate == Null)

Return;

And Change It TO:

// don't do anything if no itemtemplate

IF (_ItemTemplate == null && _alternatingItemTemplate == null)

Return;

Else IF (_ItemTemplate == Null)

_itemtemplate = _alternatingItemTemplate;

Now we are ready to build NATA1 and add a reference to the DLL to your web site project and to your toolbox in VS.NET (Warning. Make a new tab, it will add lots of new controls) Following the instructions from the authors original Article:

Step 3. Add nata1.dll to your Toolbox. Right Click your Toolbox. Choose "Add / Remove Items", Click Browse, And Find Nata1.dll. Nata1 Controls Are Now Added to your Toolbox.

there are dozens of controls, some are container controls, like ResultsRepeater, and other are for individual Items, all the ones with a smiley icon are placed in the Item or Alternating Item template, like HitUrl, HitWords, etc. You can get creative with your toolbox icons, I've included some neat ones like Homestar runner icons. Controls like QueryTime sit in the header template. some controls are specific to a search provider, eg Google has many controls, like spelling suggestions, but index server only has a COle So You Have To Be Careful To Make Sure To Be Careful To Make Sure Adding The Reference, Add The Following Line to your application_start function in your global.asax

Protected void Application_Start (Object Sender, Eventargs E)

{

Nata1.controller.start ();

}

Now when your site starts, it will check the SQL logs to see when it was last spidered, and it should begin crawling your site. You should see activity in the log, and after a while you should have a decent collection of site pages in Your Database. Now We Just Need To Build A Search Page ... (Some of this Content Is from the Original FAQ) JUST MAKE A Page Called Search.aspx for Now and Follow The Authors Original STEP 4:

Step 4: We'll Need a form to get from a search box. Go Ahead and Drag and Drop "Searchform" ONTO Any Ascx or Aspx Page In your site.

To use an image, set the SearchButtonText to an image Url (I know, not the most elegant) or enter text and make sure to set the ButtonType as well as SearchPageUrl. As you can see, there is a bug in the designer as the Image isn't Updating.

And then for step five make a page called SearchResults.aspx, you will need to link it to the control you added in step 4. and follow the Authors instructions: Step 5:. We'll build the search results page Drag and drop " Search Results Repeater "(The One with The Fairy Icon) Onto A Ascx or Aspx Page

The Two Most Import Properties Will Be "Query Provider" - Here You Want To Select Nata1. The Other Property Is Called SearchQueryTemplate Mode, Here You Want To SELECT SIMPLE.

Then Finally We Follow The Authors Step 6.

Step 6: Right Click The Template, Choose The Template You Want To Edit, And Start Dragging and Dropping Controls.

Here I Dragged The Controls SearchQuery and Totalhits Onto The Header Template, And Put An Ad Banner There Too, You Can Rotate Based On Keyword if you want.

There are several other templates you'll need to set, like NoResults, etc. There's also a template for a Search Form, and you can specify what search form controls to place there, perhaps you want an advanced search form to be at the top .

The key is in that last part ... Earlier I noted that the Repeater relies on the ItemsTemplate row, but if you add that to the control, it breaks ... so we are going to want to add some basic output to the alternating items row ... go To your html and add ...

As well as an error handler, Which cannot be set from the ui ...

An Error Has Occured and Our Support Staff Has Been Notified.

And with this, You should be able to search through the site pages in your database .. Not to pretty, but really cool, because if you made it this far you are hard core, and understand the power of what just happened on your site ... you might want to expedition the controls for the admin ... and add some noise words to your database too. Happy search ....