2

I've inherited a project that has a very odd quirk I'd like to fix. The project has a web form where user's key in data and then click a "Submit" button. Given that we have a SQL database in our office, I expected that each value would be read from the form and saved in a SQL table.

However, this is not the case. Some values are saved in SQL, but some are not. How are the values looked up that aren't saved in SQL? Simple Answer: the Javascript in the form is written so that when the user clicks submit, the mark-up of the entire form is sent to the server and saved in a text file. Then, when an authorized user wants to see how the user filled out the form, the mark-up is retrieved from the text file and sent down the wire to the browser.

This is clearly not the best way of doing this because it is wasteful of disk space and is not readily queryable.

That's why I'm rewriting the code so that future entries in the web form will be saved entirely in SQL.

In the meantime, I have tens of thousands of text files containing HTML fragments that need to be imported into SQL. My idea is to write a utility program to parse the files, one-by-one, and save the field values in SQL.

I wanted to ask for advice about the best way to do this. Normally, I use jQuery to read values from HTML, but I don't know of a practical way to do this with so many files. I also have Visual Studio 2010 at my disposal.

What tools and techniques would be best to accomplish this task?

Vivian River
  • 28,530
  • 54
  • 179
  • 298
  • 1
    This question may help: http://stackoverflow.com/questions/3340047/how-to-read-xpath-values-from-many-html-files-in-net – mccannf Nov 30 '12 at 21:29

1 Answers1

1

If you are on a Windows machine, I'd start by using windows script host to make a list of the files you need to process. If you copy the code below into a js file, save that js file into the directory containing thousands of files, then double-click the js file in Windows (to run it in wscript), it will save an array listing the thousands of files you need to check.

var io, here, fileToWrite, files;

io   =  new ActiveXObject('Scripting.FileSystemObject');
here = unescape(io.GetParentFolderName(WScript.ScriptFullName) + "\\");

fileToWrite = io.OpenTextFile(here + "\\filelist.js", 2, true);
fileToWrite.writeLine("filelist=[");

for (files = new Enumerator(io.GetFolder(here).files); !files.atEnd(); files.moveNext()) {
  var file = files.item();
  fileToWrite.writeLine('"' + file.Name + '",');
}

fileToWrite.writeLine("]");
fileToWrite.Close();
WScript.Echo("Process Complete - an array of files has been created");

You could then create a browser-based jQuery script that would load each file in sequentially, extract the information however you would normally, then save it into a more consumable format such as a text file using the FileSystemObject. (You must use IE for this.)

If you run into security issues trying to create a FileSystemObject in IE, just rename your html file with the "hta" extension - that will create an html application with the rights to do this.

Vivian River
  • 28,530
  • 54
  • 179
  • 298
TimHayes
  • 3,404
  • 18
  • 26
  • Your Javascript code works to list the files in the directory. However, I did modify your code to reflect a few best practices. Javascript treats all `var` statements as if they were a single statement at the beginning of a function. `{` goes at the end of the line, not the beginning; this is OK here, but in some cases can cause nasty surprises with the automatic comma-insertion. – Vivian River Dec 03 '12 at 16:17
  • I'm going to go ahead and accept this answer. I would have preferred that I could use jQuery without a browser, but I don't know yet if that is possible. I have posted here and elsewhere looking that that answer. – Vivian River Dec 03 '12 at 21:19
  • http://stackoverflow.com/questions/13689871/how-do-i-use-jquery-in-windows-script-host – Vivian River Dec 03 '12 at 21:21
  • 1
    You can actually use Windows Script Host to create an instance of IE and script against that. I might be able to provide some code if you can provide more details about what you want to do. – TimHayes Dec 03 '12 at 22:46
  • If you follow the link to my other SO question above, I give a more specific example of what I'm trying to do. – Vivian River Dec 05 '12 at 01:36