Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

how can return values be guaranteed? #265

Open
tripower opened this issue Jul 19, 2022 · 4 comments
Open

how can return values be guaranteed? #265

tripower opened this issue Jul 19, 2022 · 4 comments

Comments

@tripower
Copy link

tripower commented Jul 19, 2022

is it possible that a return value is guaranteed?
because if one "find" doesn not find anything the whole data is empty
so I tried this code but it only returns the first value??

const osmosis = require('osmosis');
let savedData = [];

var objSite = osmosis.get('https://www.wallstreet-online.de/etf/a0rpwh-ishares-core-msci-world-ucits-etf');
objSite.find("//*[@class='marketchange'][1]/div[1]/div[1]/div[1]/span")
    .set({'kurs': 'text()'})
    .data(data => {
        console.log("data1:" + data);
        savedData.push(data);
        objSite.find("//*[@class='marketchange'][1]/div[1]/div[3]/span")
                .set({'perc': 'text()'})
                .data(data => {
                      console.log("data2:" + data);
                      savedData.push(data);
                })
			    .done(function() {
				    var strFileData = JSON.stringify( savedData, null, 4);
				    console.log("strFileData:" + strFileData);
				    fs.writeFile('data.json', strFileData, function(err) {
					 if(err) console.error(err);
					 else console.log('Data Saved to data.json file');
				   })
				});
    });

=> output
strFileData:[
{
"kurs": "71,39"
}
]

original example which works if all xpath queries find something

    objSite.find("//*[@class='marketchange'][1]/div[1]/div[1]/div[1]/span")
    .set({'kurs': 'text()'})
	.find("//*[@class='marketchange'][1]/div[1]/div[3]/span")
	.set({'perc': 'text()'})
	.data(data => {
	  console.log("data2:" + data);
	  savedData.push(data);
	})
	.done(function() {
		var strFileData = JSON.stringify( savedData, null, 4);
		console.log("strFileData:" + strFileData);
		fs.writeFile('data.json', strFileData, function(err) {
		 if(err) console.error(err);
		 else console.log('Data Saved to data.json file');
	   })
	});

=> output
strFileData:[
{
"kurs": "71,39",
"perc": "+0,08"
}
]

@jueschus
Copy link

the first problem is that you use the same objSite object for two separate chains, so you overwrite the previous defined data/done functions (is i understand) -> you should put them together:

objSite
    .find("selector1")
    .set()
    .find("selector2")
    .set()
    .data((dataForBothSelectors) => {})
    .done()

BUT: with this solution if e.g. selector1 was not found in the DOM the chain breaks and done is called (works as implemented, s. lib/commands/find.js:42).

only solution i know would be to create multiple separate objSite objects (multiple site visits), then you can ignore if one does not resolve any value.

or is there a functionality in the library to ignore find errors?

@tripower
Copy link
Author

tripower commented Jul 21, 2022

thx

yes
would be a cool festure with a flag
that wrong xpath will return empty value instead that the whole data is empty if one fails

@tripower
Copy link
Author

my local HACK to get a result as desired
hack for osmosis

@Ruinevo
Copy link

Ruinevo commented Mar 28, 2024

you can use "do" for this

only the "do" block will fall if the selector is not found

example:

      osmosis
        .get(url)
        .delay(1000)
        .set({
          text: ['.topic-body .topic-body__content-text'],
          videoId: '.topic-body .box-external-video .box-external-video__eagle@data-vid',
          title: '.topic-body__title'
        })
        .do(
          osmosis
          .follow('.topic-body .topic-body__title-image-zoom@href')  // --> **this selector may be missing. **
          .delay(1000)
          .set({src: '.comments-page__body .comments-page__title-image@src'})
        )
        .data(resolve)
        .error(reject)
        .debug(console.log)
        .done()
    })

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants